TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning
Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View
Deep Depth Completion of a Single RGB-D Image
1st place winners of the stow task at the Amazon Robotics Challenge 2017 with MIT MCube Lab!
Matterport3D: Learning from RGB-D Data in Indoor Environments
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
Semantic Scene Completion from a Single Depth Image
Demo2 first
Multi-view Self-supervised Deep Learning for 6D Object Pose Estimation in the Amazon Picking Challenge 2016
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

Mission:

We study Computer Vision and Robotics, focusing on the computational principles underlying Artificial Intelligence. We are interested in building robots that automatically understand and interact with the physical worlds, both inferring the semantics and extracting 3D structure.

We design end-to-end algorithms to learn deep 3D representations from big 3D data for visual scene understanding. We believe that it is critical to consider the role of a machine as an active explorer in a 3D world, such as a robot, and learn from rich 3D data close to the natural input to human visual system.

Specifically, our group is at the frontier of 3D Deep Learning, RGB-D Recognition and Reconstruction, Deep Learning for Robotics, Place-centric 3D Context Representation, Synthesis for Analysis, Big Data Robotics, Autonomous Driving, Robot Learning, Large-scale Crowd-sourcing, and Petascale Big Data.

    News:

We gratefully acknowledge the generous support of Intel, Amazon, Ford, ABB, National Science Foundation, Google, MERL, Facebook, Microsoft, and NVIDIA for our research.