In an assembly situation, the following intelligent abilities of man are necessary, i.e. that he
The fact that all these abilities are produced in a certain situation, i.e. that man is thereby situated, drastically limits his fundamental possibilities: the spoken word can only have certain meanings, as it refers to the actual situation; only certain own utterances are possible, as they must refer to both the task to be solved and the situation; for this reason, only certain actions are possible. In this way, situatedness is a prerequisite for a more exact examination of the intelligence abilities of man as well as for the transmission of the insights gained to artificial systems. In the foreseeable future, we will have to depart from the vision of the universally employable robot and aim at specialised robots for limited tasks.
The remarkable thing about the abovementioned intelligence abilities of man is that he can produce these even when the information available, i.e. what he has seen or heard, is incomplete or garbled. This ability is defined as robustness. Many artificial systems, robots or computer programmes suffer from the very fact that they are not comparatively robust. This characteristic in man, therefore, is of great interest to our Collaborative Research Centre. This characteristic originates, in part, from the fact that humans can relate information from various sources, i.e. that which they have seen or heard, and process the whole integrated. Disorder or incompleteness in the spoken word can be counterbalanced by that which is seen and vice- versa. Apart from this, that which is spoken can contain directions to look more closely (active vision). In the same way, we use properties, which we perceive in objects, e.g. their colour, their form, their size and their position, in order to refer to these objects in speech. This is the reason, for example, why we refer to an object as a small red screwdriver in the tool box.
As the situatedness of natural and artificial communicators plays a decisive role in the overall concept of the Collaborative Research Centre, it was necessary to select a reference situation, which touches all projects. This situation can be defined as follows: two communicators must cooperatively construct an object. One, the instructor, has a diagram and instructs the other, the constructor, in such a way that the constructor, with the help of the spoken directions, can carry out the necessary actions as unproblematically as possible. Such an object can be a model aeroplane, which is to be constructed from the components of a wooden building kit. In this way, it is possible to make the situation to be described sufficiently interesting, but also sufficiently controllable and practically manageable. The ability to control is necessary, because our knowledge on the abilities of man in such a situation is still very fragmentary. Controlled experiments must be carried out, as man is to serve as the model for the machine constructor. The practical manageability is necessary, because the technical means, e.g. for recognition of the spoken word or for understanding a situation based on a diagram, are still a long way short of the abilities of man. In spite of this, our Collaborative Research Centre aims at constructing concrete, but limited artificial systems in a step by step procedure. Processes must be developed which will enable cooperation between different specialised systems for individual intelligent abilities.
In the thematic field A Speech and Visual Perception three projects examine how an artificial communicator can receive, process and understand acoustic and visual information via sensors, i.e. microphone and camera. Here, knowledge of the most varied form is used, which must be organised in a certain manner and brought into relation to each other (Project A1). It must further be clarified how, based on the raw data of the image, i.e. information on the individual image pixels, objects can be identified and defined by the others (Project A2). Studies are being carried out on the processing of the spoken word: what contribution words such as so, yes, nope, ehm etc., intonation and speech melody make towards comprehension in a directional dialogue (Project A3).
In the thematic field B Perception and Reference four projects examine how the optically available information on the one hand and the spoken information on the other are to be put into relation with each other. Only in this manner can an artificial communicator correctly identify an object on hand of a spoken formulation such as the small red screwdriver in the tool box (Project B3) or select an appropriate formulation, so that his partner knows, which object is meant (Project B2). Experiments on how humans refer to objects with formulations and how the link between speech and image processing in a technical sense are a prerequisite for this (Project B1). Processes of active vision or "looking more closely" are examined with the help of a computer/camera system for recording eye movements (Project B4).
In the thematic field C Knowledge and Inference one project examines how the knowledge, of which communicators avail, allows only certain conclusions in a given situation. Our investigations examine how the connection between a word and an object, which we wish to define with the word, can continually alter in an on-going assembly task. For example, a wooden disc from the assembly kit is first referred to as a wooden disc and later as the wheel of an aeroplane (Project C1).
In the thematic field D Speech-Action Systems two projects examine the question of according to which principles the integration of individual intelligence abilities occur. To achieve this, two complementary procedures are examined, but a decision has yet to be made on their general capacity. On the one hand, for situations, which are even more limited than those already described, we try - in one step - to bridge the gap between the understanding of an utterance and the resulting action (Project D1). On the other hand, based on the systems in the other projects, it is attempted to collate the functional units and to examine the performance capacity of differents forms of the collation through computer experiments (Project D3).
The theme of the Collaborative Research Centre, in which Linguistics, Cognitive Science and Artificial Intelligence research are closely entwined, is unique in this form. The scientific headquarters at the University of Bielefeld, in North Rhine Westphalia, Germany has succeeded in entering a field of research, which shows the way for the future and which, in the trend of the growing importance of intelligent information systems is a decisive factor for technical innovation.
University of Bielefeld
SFB 360
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
or by emailing to: anke@SFB360.Uni-Bielefeld.DE (Anke Weinberger)
The WWW homepage for SFB 360 is
http://www.sfb360.uni-bielefeld.de/
More general information, abstacts of reports, and an ftp-server address for postscript versions of reports can be accessed from this WWW page.
to SFB Homepage
|
created by: Anke Weinberger (1996-02-06). maintained by:Anke Weinberger (2000-09-07). |