In a recent blog post, Facebook has announced they have open-sourced AI Habitat, an Artificial Intelligence (AI) simulation platform that is designed to train embodied agents, such as virtual robots. Using this technology, robots can learn how to grab an object from an adjacent room or assist a visually-impaired person in navigating an unfamiliar transit system.
The technology leverages embodied AI which focuses on interactive environments to train real-world systems. This is a different approach than relying upon static data sets which other researchers have traditionally used.
A team of Facebook researchers, including Manos Savva, Abhishek Kadian, Oleksandr Maksymets and Dhruv Batra, have released a research paper that demonstrates the capabilities of Al Habitat. Facebook decided to open-source this technology to move this subfield forward. Savva, a Facebook researcher, further explains:
Our goal in sharing AI Habitat is to provide the most universal simulator to date for embodied research, with an open, modular design that’s both powerful and flexible enough to bring reproducibility and standardized benchmarks to this subfield.
Facebook AI has previously explored embodied AI when it created agents to navigate the streets of New York City. This previous work leveraged natural language processing (NLP), computer vision (CV) and reinforcement learning (RL) to communicate with humans. However, there were some challenges with their previous initiatives, Savva explains:
While our work mirrored the larger state of embodied AI research — with no standard platform available to easily run experiments and measure results against others, new projects often required starting from scratch. Given the resource-intensive nature of building and rendering simulated environments, this made progress slower compared with subfields whose standard benchmarks — such as ImageNet and COCO — enable collaboration and rapid iteration.
The first layer in the AI Habitat stack is the simulator which includes a Replica dataset which is available on GitHub, provided by Facebook Reality Labs (FRL). The simulator includes realistic 3D reconstructions of a staged apartment, retail store and other indoor spaces. The benefit of providing this level of details, reduces the training gap between virtual and physical spaces.
Image source: https://ai.facebook.com/blog/open-sourcing-ai-habitat-an-simulation-platform-for-embodied-ai-research/
In addition to leveraging these 3D data sets, the simulator is able to render assets quickly. Savva characterizes the performance of AI Habitat to traditional methods as:
In benchmark tests, Habitat-Sim with multiple processes renders detailed scenes at 10,000 frames per second (FPS) on a single GPU, compared with a typical rate of 100 FPS on other simulators. That kind of speed boost is relevant to vision-based embodied learning, since increasing the number of frames that an agent experiences in a given period can directly increase training efficiency.
The second layer in AI Habitat’s architecture is the Habitat-API which exposes tasks such as visual navigation, question/answering through an API. This API is decoupled from the simulator and provides developers with more modularity as they adopt different components in the AI Habitat stack.
Within the third layer in the AI Habitat stack, developers have the ability to specify training and evaluation parameters that allow for modification of complexity settings and indicate which metrics to focus on.
AI Habitat is currently available and had been tested at a recent autonomous navigation challenge and was used at the Habitat Embodied Agents workshop at CVPR 2019. Moving forward, Facebook anticipates introducing realistic avatars which will enable collaboration with another avatar as you communicate ideas and reactions.