Here are some more details, below (or see a Twitter thread with attached media: https://twitter.com/i/status/1729515379892826211 or https://nitter.net/i/status/1729515379892826211):

We engineered Dobb·E to maximize efficiency, safety, and user comfort. As a system, it is composed of four parts: a data collection tool, a home dataset, a pretrained vision model, and a policy fine-tuning recipe.

We teach our robots with imitation learning, and for data collection, we created the “Stick”, a tool made out of $25 of hardware and an iPhone.

Then, using the Stick, we collected a 13 hour dataset in 22 New York homes, called Homes of New York (HoNY). HoNY has 1.5M frames collected over 216 different "environments" which is an order of magnitude larger compared to similar open source datasets.

Then we trained a foundational vision model that we can fine-tune fast (15 minutes!) on a new task with only 5 minutes (human time)/ 90 seconds (demo time) of data. So from start to finish, it takes about 20 minutes to teach the robot a new task.

Over a month, we visited 10 homes, tried 109 tasks, and got 81% success rate in simple household tasks. We also found a line of challenges, from mirrors to heavy objects, that we must overcome if we are to get a general purpose home robot.

We open-sourced our entire system because our primary goal is to get more robotics and AI researchers, engineers, and enthusiasts to go beyond constrained lab environments and start getting into homes!

So here is how you can get started:

1. Code and STL files: https://github.com/notmahi/dobb-e/

2. Technical documentation: https://docs.dobb-e.com/

3. Paper: https://arxiv.org/abs/2311.16098

4. More videos and the dataset: https://dobb-e.com

5. Robot we used: https://hello-robot.com