The key in building our RoboBrain is to be able to learn from a variety of multi-modal data and from many types of learning signals. Some examples are:
A robot has to learn from multi-modal data sources such as videos, images, language and even physical interaction with humans. This necessitates building a large-scale knowledge graph that allows efficient data storage, probabilistic queries and retrieval. Another challenge is the variety of learning algorithms the system should support, from large-scale offline learning to real-time online learning.
This multi-modality of data and the large probabilistic query space makes this problem more challenging than existing large-scale systems that typically store single type of data source such as images, or textual data.
For seamless human-robot collaboration robots should be able to interact with humans using natural language. Robots should not only parse the language, but also ground its meaning into suitable actions based on the context and the environment.
This require building learning algorithms that can model the ambiguity in language, the uncertainty in the robotic actions, and the implicit representation of the task and the environment. This will require exploring new ways for collecting natural language data from humans, and designing algorithms for mapping the language to robot actions obeying environment constraints (the laws of physics).
In order to act in the environment, a robot should be able understand its surrounding objects, furniture layout, humans, road-signs, etc. This requires building data-driven vision algorithms that run in real-time.
The challenges encountered here are far richer than traditional computer vision because:
Robots need to reason about the constraints and behavior of the agents in the real world. We are developing methods that model other human's intentions, methods that learn constraints by observing events in the environments. Unlike other traditional rule-based systems, the challenge here is to learn and probabilistically integrate such knowledge into our Robo Brain.
We are using our Robo Brain in various robots such as Baxter, PR2, and several others. Since every robot is physically different and operates in a variety of environments, we need to build methods that allow our Robo Brain to be embodied in different robots and for different tasks.
Cloud Robotics has potential to improve performance in at least five ways: