feat: add unified dual-arm VR teleoperation - #3392
Open
TomCC7 wants to merge 24 commits into
Open
Conversation
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## krishna/feat/openarm-damiao #3392 +/- ##
===============================================================
+ Coverage 75.92% 76.46% +0.53%
===============================================================
Files 1205 1201 -4
Lines 117065 115802 -1263
Branches 10425 10334 -91
===============================================================
- Hits 88887 88553 -334
+ Misses 25417 24254 -1163
- Partials 2761 2995 +234
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 46 files with indirect coverage changes 🚀 New features to boost your workflow:
|
TomCC7
commented
Aug 7, 2026
# Conflicts: # CONTEXT.md # dimos/cli/dimos.py # dimos/control/tasks/cartesian_ik_task/cartesian_ik_task.py # dimos/control/tasks/cartesian_ik_task/test_cartesian_ik_task.py # dimos/control/tasks/quest_teleop_ik_task/test_teleop_task.py # dimos/control/tasks/teleop_task/teleop_task.py # dimos/hardware/test_adapter_registries.py # dimos/manipulation/planning/world/roboplan_world.py # dimos/manipulation/test_roboplan.py # dimos/robot/all_blueprints.py # dimos/robot/manipulators/common/blueprints.py # dimos/robot/manipulators/common/mixed.py # dimos/robot/manipulators/openarm/blueprints/teleop.py # dimos/robot/manipulators/openyam/blueprints/teleop.py # dimos/robot/manipulators/piper/blueprints/teleop.py # dimos/robot/manipulators/xarm/blueprints/teleop.py # dimos/teleop/quest/README.md # dimos/teleop/quest/test_quest_teleop_module.py # docs/capabilities/manipulation/index.md
…o cc/feat/dual-arm-teleop # Conflicts: # dimos/cli/dimos.py # dimos/hardware/test_adapter_registries.py # dimos/manipulation/test_roboplan.py
TomCC7
commented
Aug 8, 2026
TomCC7
marked this pull request as ready for review
August 8, 2026 06:34
TomCC7
requested review from
Dreamsorcerer,
leshy,
mustafab0,
paul-nechifor and
spomichter
as code owners
August 8, 2026 06:34
TomCC7
commented
Aug 8, 2026
TomCC7
commented
Aug 8, 2026
TomCC7
commented
Aug 8, 2026
github-actions
Bot
added
the
ready-to-merge
ManipulationModule._control_coordinator: ControlCoordinator resolved by exact class identity, so OpenArmTeleopCoordinator (a ControlCoordinator subclass added to gate real/mock hardware on CAN flags) never matched. The ref silently became None, and viser Plan->Execute crashed hardware with AttributeError: 'NoneType' object has no attribute 'execute_trajectory'. Reproduced live on the OpenArm 2.0 rig. Match module-type refs by issubclass instead of identity in both the blueprint-time resolver and get_instance()/_instance_keys_of(). Ambiguity handling (multiple matching instances -> "pass the instance name") is unchanged; this only rescues subclass providers that were false negatives before. Added a regression test reproducing the exact failure with a Calculator1 subclass; verified it fails with the pre-fix identity check and passes with issubclass.
teleop_ik_task's max_joint_velocity_rad_s defaulted to 5.0 rad/s (~286 deg/s) uncapped by the OpenArm quest blueprint. Krishna reported the real OpenArm 2.0 felt too sensitive under Quest teleop. Set to 2.0 rad/s (~114 deg/s), tuned on hardware (1.0 felt too slow).
github-actions
Bot
removed
the
ready-to-merge
github-actions
Bot
added
the
ready-to-merge
github-actions
Bot
added
ready-to-merge
TomCC7
changed the title
Measured feedback beyond feedback_limit_tolerance raised and skipped the control tick entirely, resetting the command trajectory. On real OpenArm 2.0 hardware a joint resting a fraction of a degree past its limit (mechanical stop give, encoder noise) could never recover: the tick that would compute a corrective command was the exact one being skipped. Reproduced live, right_arm/joint6 stuck until manually flexed off its stop. Add feedback_clamp_margin (0.05 rad default). Deviations within it now clamp to the limit and the tick proceeds normally; only a deviation beyond it still raises as a genuine fault. The downstream command envelope already bounded commands safely, so removing the premature raise is sufficient. Threaded through all four task configs that share this Pink core: pose_target_ik, teleop_ik_task, cartesian_ik_task, eef_twist_task. Verified on hardware: the same joint-limit condition that previously deadlocked now self-corrects without manual intervention.
github-actions
Bot
removed
the
ready-to-merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
Quest arm teleoperation assumed one controller, one end-effector, and one IK task. OpenArm needs both controllers to drive one bimanual model in a single solve. The existing Cartesian, twist, and teleop paths also duplicated Pink setup and lacked a common streaming safety envelope for noisy, delayed hardware feedback.
Solution
PoseTargetIKTaskcore.CartesianIKTask,EEFTwistTask, andTeleopIKTaskremain thin input-specific leaves.TeleopIKTask, including optional gripper joints. A two-hand task captures both references together and requires both primary buttons; timeout, release, preemption, or E-stop clears the full session.PinkIKexposes the planning API,PinkPoseTargetSolverowns streaming control state, and both use the private shared Pink solver core.PinkPoseTargetSolversubclass. OpenArm tunes frame costs, posture weights, and per-arm manipulability tasks without replacing common solving or safety logic.TeleopControlCoordinatorfor teleop-only pose and button ports while leaving the generic coordinator free of Quest streams. Existing A1Z, mixed-arm, OpenYam, Piper, and XArm Quest blueprints use the unified task configuration.teleop-quest-openarmwith one bimanual IK task and one OpenArm URDF. It uses fake hardware by default; specifying both CAN interfaces selects real hardware. The blueprint also includes the manipulation planner, Viser visualization, and a higher-priority dual-arm trajectory task.How to Test
Run the complete feature with fake hardware:
To use real OpenArm hardware, specify both CAN interfaces:
Run the focused control, Pink, routing, Quest, and OpenArm tests:
Current result: 156 passed.
AI assistance
Codex with GPT-5 assisted substantially with architecture exploration, implementation, tests, documentation, and verification. The author developed and reviewed the changes interactively.
Checklist