Retarget two arms with Bridge

CERES Bridge drives the two XLeRobot arm models from Quest wrist motion. A CPU inverse kinematics loop updates both arms while Foxglove displays the robot, head and hands, live camera and waveforms. The example runs without Isaac Sim. An optional IsaacTeleop adapter supplies wrist targets through NVIDIA's retargeting API.

Start the demo

Install the receiver on Ubuntu 24.04 using the receiver setup, then install the visualisation dependencies from the receiver directory:

python -m pip install '.[worker,foxglove,teleop]'
ceres-bridge listen --name dual-arm-demo

Pair Bridge on Quest with the displayed code and select Start streaming. In a second terminal in the same Python environment:

ceres-bridge foxglove --robot xlerobot --robot-rate 60

Open the printed Foxglove connection link and import the layout from the printed download address. The dual-arm layout is also included in the source distribution. For VP8 video, import the dual-arm VP8 layout.

For Foxglove on another computer, add --host <receiver-LAN-address>. The existing --assets <directory> option adds the locally installed Quest and hand meshes to the acquisition view.

Workspace

View	Display
Robot in 3D	XLeRobot cart, both arms, articulated grippers, wrist cameras and wrist targets
Acquisition in 3D	Quest head and hand poses with live camera projection
Outward camera	Independent live Quest video
Wrist position	Left and right wrist X, Y and Z in metres
Wrist rotation	Left and right roll, pitch and yaw in radians
Left arm joints	Five arm angles and gripper angle
Right arm joints	Five arm angles and gripper angle
Frame rates and load	Received camera, head and hand rates, robot update rate and visualiser process CPU

The performance tab adds resident memory, viewer loop time, inverse kinematics time, pose age and clock uncertainty. Both 3D panels enable Foxglove's rendering statistics. Pose and robot waveforms share a sliding 15-second window.

Mapping the hands

Each arm follows its wrist's absolute pose through one fixed mapping from the CERES origin to the robot base. Bridge first converts WebXR poses into X forward, Y left and Z up. --position-scale scales each wrist position, with a default of 0.6. --robot-yaw rotates that position about Z and --robot-origin X Y Z adds a translation in metres. The same yaw rotates the absolute wrist orientation. Thumb-to-index distance opens and closes the corresponding gripper.

The default origin is 0 0 0 and the default yaw is zero degrees. To rotate the CERES frame by 90 degrees and place its scaled origin 0.2 metres above the robot base:

ceres-bridge foxglove --robot xlerobot --position-scale 0.6 --robot-origin 0 0 0.2 --robot-yaw 90

The position mapping is robot_origin + Rz(robot_yaw) (position_scale wrist_position). Both arms use this same fixed transform.

The gripper's approach follows the fingers and palm roll turns the gripper about that approach. WebXR wrist coordinates point towards the fingers along -Z, which becomes +X after the Bridge basis conversion. The XLeRobot gripper approaches along its local -Y axis, so the rotation target is Rz(robot_yaw) wrist_rotation Rz(90 degrees). This fixed local alignment changes orientation only and leaves the target position unchanged.

The model uses the XLeRobot URDF and meshes. The receiver includes all 20 visual links as 12 shared GLB assets, preserving their original geometry, visual origins, scales and materials. Foxglove loads these assets from the receiver and caches them while link poses update. The cart and head remain stationary, while both arms, grippers and wrist cameras follow the commanded joints. Each arm has five motion joints and one gripper hinge.

The solver fits position first, then fits wrist orientation using the remaining joint freedom. An incompatible rotation cannot pull the arm away from its position target. Beyond reach, the solver retains its best position and continues fitting wrist roll within a fixed numerical position tolerance. It publishes the remaining position and rotation error. Joint angles obey the model limits and move at up to 2 radians per second.

During a tracking gap, each arm continues towards its last absolute target for 0.5 seconds, reporting coasting. --tracking-grace sets this duration in seconds. If tracking remains absent, the affected arm returns to its neutral home configuration within the same joint speed limit. Losing both hands returns both arms to neutral. The arm reports returning while moving home and neutral when it arrives. When tracking resumes, the next absolute wrist pose determines its target through the same fixed transform. Connection and reference-space epochs retain this mapping. Each source observation replaces the target once, while the servo continues towards that target between observations. The robot consumer has its own receiver connection and one outstanding solve, while video and Foxglove geometry refresh independently.

Python application

The Python example exposes the same retargeter for integration with an application:

from ceres_bridge import Receiver
from ceres_bridge.teleop import DualArmTeleop
import numpy as np

robot_from_ceres = np.eye(4)
robot_from_ceres[:3, 3] = [0, 0, 0.2]
robot = DualArmTeleop(position_scale=0.6, robot_from_ceres=robot_from_ceres)
with Receiver(video=False) as receiver:
    state = robot.update(receiver.latest())
    joint_names = state["joint_names"]
    joint_positions = state["joint_positions"]  # Radians, in joint_names order.

robot_from_ceres is a fixed 4 x 4 rigid transform, with an identity matrix as its default. Position scaling applies before this transform. state["arms"] contains per-arm targets, current end-effector poses, tracking state, residuals and solver time. robot.reset() clears observation history and starts both arms' return to neutral while retaining the fixed mapping for the next tracked pose. robot.link_transforms() returns forward kinematics in ceres_robot_base.

robot.urdf returns a self-contained visual URDF with the same upstream joint tree and embedded GLB meshes. Its base and head joints are fixed at their displayed poses. Use Foxglove's default Y-up mesh setting for these standard glTF assets. The package also includes the unmodified source URDF, asset manifest, source hashes and Apache 2.0 licence under ceres_bridge/data/xlerobot. The visual export contains visual geometry only.

IsaacTeleop adapter

Install NVIDIA's IsaacTeleop retargeting package in the application environment, then select it:

python -m pip install 'isaacteleop[retargeters-lite]'
ceres-bridge foxglove --robot xlerobot --retargeter isaacteleop

The adapter maps CERES joint names into IsaacTeleop's hand input and calls Se3AbsRetargeter for each wrist. It shares the fixed absolute mapping, gripper and CPU inverse kinematics implementation with the default adapter. Headset capture continues through CERES Bridge.

Run python examples/dual_arm_benchmark.py --retargeter isaacteleop to measure the combined adapter and two-arm solver on the receiver host. Use --retargeter cpu for the default path.

Topics and measurements

Topic	Fields
/ceres/head/motion, /ceres/left/motion, /ceres/right/motion	position.x/y/z, rotation.roll/pitch/yaw, tracked
/ceres/robot/joints	left and right objects containing shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll and gripper
/ceres/robot/scene	Current robot geometry in ceres_robot_base
/ceres/robot/diagnostics	update_fps, solve_ms, per-arm tracking, status and pose residuals
/ceres/diagnostics	Incoming rates, age, process_cpu_percent, process_rss_mb and loop_ms

Robot joint waveforms show the current retargeted model configuration in radians. update_fps counts active servo updates, including continued motion between wrist observations and the return to neutral. solve_ms covers both arm solves. CPU load measures the Foxglove output process, including its robot thread, with one core represented by 100 percent. Resident memory is measured in MiB. loop_ms measures a viewer iteration, including its receiver read. Tracking gaps break the motion traces and rotation traces unwrap across half turns. The topic specification defines the gap representation.