The hardest problem in modern robotics isn’t the model. It’s the data.
Over the past two years researchers have shipped increasingly capable policy architectures — transformer-based vision-language-action models, diffusion and flow-matching policies, even learned world models — and the GPUs to train them are abundant. What’s missing is large, diverse, real-world manipulation data: the thousands of hours of demonstrations that teach a robot how to actually pick up, place, fold, and assemble things in the messy physical world.
That gap is the explicit motivation behind Grabette, an open system Hugging Face released this week (Hugging Face blog). The pitch is deliberately humble. You don’t need a robot to collect robot data. You need a human hand, a simple gripper, a camera, and a way to recover the six-degrees-of-freedom (6-DoF) trajectory of what the hand did. Capture the demonstration and you have something a machine can learn from.
Why teleoperation stopped scaling
The conventional way to build manipulation datasets is teleoperation: a person drives a real robot through a task while sensors log every joint angle and camera frame. It works, but it doesn’t scale. Teleop rigs are expensive, they tie up physical hardware, and collecting a meaningful volume of demonstrations is tedious enough that most labs cap out at a few hundred episodes per task.
Grabette inverts the setup. Instead of operating a robot, you perform the task yourself with a lightweight handheld gripper. The device records what your hand does and reconstructs a clean, robot-ready dataset — no robot on site, no motion-capture volume, no lab booking.
The idea isn’t new. Grabette is directly inspired by the Universal Manipulation Interface (UMI) from Stanford (UMI project), a handheld gripper with a fisheye camera that records demonstrations “in the wild,” recovers camera trajectories with SLAM, and trains visuomotor policies from them. UMI proved the recipe works. A handful of closed-source devices followed — Agibot’s MEgo gripper, Genrobot’s DAS gripper, and the Sunday Robotics skill-capture glove — but Grabette’s authors say the goal was to drive the barrier from “I have a task” to “I have a trained model” as low as possible, and to keep the entire stack open.
What’s actually in the box
Grabette is a handheld gripper instrumented with everything needed to reconstruct a manipulation demonstration. It carries two cameras with deliberately separate jobs: a cheap, wide fisheye lens gives the policy the context-rich, wrist-camera-style view it needs, while an RGBD camera does the heavier lifting of robust 6-DoF tracking. Magnetic encoders capture the gripper’s joint state.
Crucially, the components are ordinary. A Raspberry Pi, a standard Pi camera, an off-the-shelf OAK-D depth camera, magnetic encoders — the whole point is that anyone can build one from parts they can simply order. There’s no closed pipeline and no fork lock-in.
Grabette is also robot-agnostic by design. Nothing in the capture or the data format assumes a particular arm. Demonstrations are stored as camera-local 6-DoF cartesian pose plus gripper state, and the output is a standard LeRobot dataset published on the Hugging Face Hub (LeRobot). The same data can therefore drive different robots and different learning methods — though you’ll still need the matching Gripette gripper mounted on your arm to act it out.
Gripette is Grabette’s robotic twin: the arm end-effector that executes the movements the handheld device recorded. Together they close the loop from human demonstration to machine execution.
From a hand demo to a training set
The release is built to take a first-time user from “I want to demonstrate a task” to “I have a training-ready dataset” without prior expertise.
You press a button and the observation camera, the tracking camera (color, depth, and IMU), and the gripper’s encoder joint values are recorded simultaneously, locked to a single shared clock so everything stays synchronized. Press again to end the episode, and the data is saved locally on the Raspberry Pi. Then you open the Grabette dashboard in a browser, select the episodes you want in the dataset, and one click kicks off post-processing. From there, the data is ready to start training.
None of the processing requires a local install. The pipeline runs from the browser and posts the result straight to the Hub, sitting naturally inside the modern open robotics stack: LeRobot for the dataset format, the Hugging Face Hub for sharing, and a browser-based tool for conversion.
To show the loop end to end, the project ships a complete example built from 200 recorded demonstrations. The authors are careful to frame it as a demonstration of what the system enables, not the product itself — the release is the recording system, and your Grabette data works with any method that consumes LeRobot datasets.
The community bet
A dataset is only interesting if it trains something, and the data bottleneck won’t be solved by one lab. Grabette’s larger ambition is to make recording a demonstration as easy as shooting a video, so that anyone can contribute. Build a Grabette, record the tasks you care about, and share them on the Hub. Every episode makes the open dataset bigger and more diverse; every contributor makes robot learning a little less gated behind expensive hardware.
The project is only getting started. A head-mounted POV device called Casquette is in the works to complement Grabette with egocentric capture. But the most important next step, the team says, isn’t theirs alone — it’s for people to start recording, and to build the dataset together.
