To successfully train robotic systems through Learning from Demonstrations, large and high-quality datasets of human demonstrations are normally required. However, dual-arm data collection is severely bottlenecked by prohibitive hardware costs, complex dual-arm coordination, and a persistent risk of self-collisions, which restricts the deployment of these platforms. In this work, we present ADAPT (ADAM Dual-Arm Platform for Teleoperation), a comprehensive hardware and software framework based on GELLO that enables safe, intuitive, and low-cost data collection for the ADAM mobile manipulator. The proposed system incorporates an active weight compensation mechanism to bridge the physical discrepancies between the leader device and the robot, alongside an active safety system based on Signed Distance Fields that prevents self-collisions without restricting the operator's workspace. The framework has been evaluated in both simulated environments and on the real ADAM platform, demonstrating that inexperienced users can safely collect complex dual-arm demonstrations.
ADAPT (ADAM Dual-Arm Platform for Teleoperation) is a comprehensive hardware and software framework designed to democratize and secure the collection of high-quality dual-arm data through Learning from Demonstration (LfD).
Based on the low-cost philosophy of GELLO, our method solves the three major challenges of dual-arm teleoperation: prohibitive costs, operator fatigue, and the constant risk of self-collisions. The workflow is divided into three real-time interconnected pillars:
In short: ADAPT acts as a smart teleoperation shield. It translates the user's natural movements, validates them in a digital twin using SDF to prevent self-collisions, and executes 100% safe trajectories on the real robot.
To evaluate the ADAPT framework, the experimental setup encompassed a diverse spectrum of dual-arm coordination paradigms. Tasks were designed to exploit true dual-arm synergy, solving operations that are impossible or inefficient for a single arm. The evaluation scenarios fall into 4 categories:
A concurrent pick-and-place scenario where each arm independently relocates separate items. This demonstrates spatial synergy, relying on the system's collision avoidance to maximize productivity in a shared workspace.
Involving handover and liquid pouring tasks. These represent a leader-follower dynamic where one arm provides stability (e.g., holding a glass) while the other executes a precision action (e.g., pouring).
Transporting large, rigid objects like bulky boxes. This requires synchronous control over a single payload, ensuring both manipulators maintain precise trajectory coordination to prevent dropping the object or inducing mechanical strain.
Handling flexible materials like cables and textiles. These objects deform under gravity and require simultaneous dual-point tension control, highlighting a core advantage of dual-arm teleoperation.
Select a task and a user to load the demonstration video.
The ADAPT framework incorporates an active safety system based on Signed Distance Fields (SDFs) to prevent self-collisions without restricting the operator's workspace. By evaluating the robot's joint states in real-time within the digital twin, the SDF algorithm provides haptic repulsion that halts dangerous movements, guaranteeing risk-free data collection during unconstrained dual-arm teleoperation.
Real-time haptic repulsion preventing dual-arm self-collision and object collision. When the SDF detects the collision, the system blocks the arm to avoid real collision.
The ADAPT framework leverages high-fidelity simulated environments (based on ADAMSim) to streamline the data acquisition process for Imitation Learning. By utilizing a digital twin of the dual-arm setup, operators can perform complex teleoperation tasks in a safe and controlled virtual space. This approach not only prevents physical wear and accidental collisions during the initial stages of data collection, but also enables rapid iteration and the generation of diverse datasets across varied scenarios, ensuring robust policy training before real-world deployment.
Teleoperation of the dual-arm system within the digital twin. The virtual environment replicates the physical kinematics and dynamics, allowing the operator to safely collect manipulation data without hardware risks.
