Universität Bielefeld - Sonderforschungsbereich 360

Architectural Design Issues for Man-Machine Cooperation - Integrating Speech and Vision for Human Instructed Assembly

Nils Jungclaus, Helge Ritter, Gerhard Sagerer
University of Bielefeld

Architectures proposed for artificial systems mostly focus on autonomous systems carrying out tasks in a rather narrow domain. Flexible man-machine cooperation tends to be more much more open-ended and can benefit greatly from an efficient coordination of multi-modal user interfaces.

We present an approach to a system architecture integrating speech and image recognition to follow human instructions in a simplified assembly scenario. The main architectural aspects focus on data-flow, control-flow and handling of different time requirements of modules operating at various levels within a hybrid system. In particular, we propose a generic memory module to coordinate bottom-up data-driven and top-down expectation-driven evaluation as well as focus information. The practical realization is based on a flexible communication tool (DACS) that provides a carefully chosen set of communication primitives that can be used in parallel and allow the integration of existing system modules easily.

Anke Weinberger, 1997-12-03