Towards Integrating Learning by Demonstration and Learning by Instruction in a Multimodal Robotics

Stefan Wermter, Mark Elshaw, Cornelius Weber, Christo Panchev, Harry Erwin

Research output: Chapter in Book/Report/Conference proceedingConference proceedingpeer-review


Learning by demonstration and learning by instruction offers a potentially more powerful paradigm than programming robots directly for specific tasks. Learning in humans or primates substantially benefits from demonstration of actions or instruction by language in the appropriate context and there is initial neurocognitive cortical evidence for such processes. Cortical assemblies have been identified in the cortex that activate in response to the performance of motor tasks at a semantic level. This evidence supports that such mirror neuron assemblies are involved in actions, observing actions and communicating actions. Furthermore, neurocognitive evidence supports that cell assemblies are activated in different regions of the brain dependent on the
action type being processed. Based on this neurocognitive evidence we have begun to design a neural robot in the MirrorBot project that is based on multimodal integration and topological organisation of actions using associative memory. As part of these studies in this paper we describe a self-organising model that clusters actions into different locations dependent on the body part they are associated with. In particular, we use actual sensor readings from the MIRA robot to represent semantic features of the action verbs. Furthermore, ongoing work focuses on integration of
motor, vision and language representations for learning from
demonstration and language instruction.
Original languageEnglish
Title of host publicationProceedings of the IROS-2003 Workshop on Robot Learning by Demonstration
Number of pages8
Publication statusPublished - 2003
Externally publishedYes
EventInternational Conference on Intelligent Robots and Systems: Workshop On Robot Programming Through Demonstration - Las Vegas, United States
Duration: 27 Oct 200331 Oct 2003


ConferenceInternational Conference on Intelligent Robots and Systems
Abbreviated titleIROS 2003
Country/TerritoryUnited States
CityLas Vegas


Dive into the research topics of 'Towards Integrating Learning by Demonstration and Learning by Instruction in a Multimodal Robotics'. Together they form a unique fingerprint.

Cite this