To allow humanoid robots to operate in real environments they must be capable of dealing with novelty, uncertainty and unforeseen situations. The ultimate goal of the project was the design of a cognitive system capable of performing grasping and manipulation tasks in open-ended environments. This work focused on the perception issues in such a system. One outcome was a novel technique for the classification of 3D data where Gaussian processes were used to infer spatial occupancy by combining stereo and tactile sensing with a humanoid hand. To aid in the practical use of robots in domestic applications, another platform was developed with natural language processing. This system allowed for the interactive segmentation and recognition of household objects from stereo data. The underlying processing relied on a Markov random field framework to perform sensor fusion between speech and vision.

Associated Publications:
 

• Mind the Gap - Robotic Grasping under Incomplete Observation, International Conference on Robotics and Automation, 2011
• Attention Based Active 3D Point Cloud Segmentation, International Conference on Intelligent Robots and Systems, 2010
• Strategies for Multi-Modal Scene Exploration, International Conference on Intelligent Robots and Systems, 2010
• Acting and Interacting in Natural Environments, IROS Workshop on Semantic Mapping and Autonomous Knowledge Acquisition, 2010
• Enhanced Visual Scene Understanding through Human-Robot Dialog, AAAI 2010 Fall Symposium: Dialog with Robots, 2010
• Active Scene Analysis, Robotics: Science and Systems Workshop: Towards Closing the Loop: Active Learning for Robotics, 2010