A robot policy that succeeds in MuJoCo or Isaac Lab gives you a reason to run the next experiment. The hardware test should answer a precise question: which part of that behavior survives contact with the real setup?
You might be developing a learned policy, a hand-written controller or a robot project with a coding agent. The useful next step is the same: define a small test whose observations can change your decision. A convincing animation alone cannot answer whether the intended robot can reproduce it.
1. Choose one question before choosing a robot
“Validate my model” is too broad to make a useful booking. Choose the observation that would affect your next engineering decision. For example: does the gripper retain this object during a slow transfer, or does the policy still complete its task when the camera moves slightly?
Separate the behavior from the explanation. An object slipping is an observation. Incorrect friction, insufficient grip, camera error and a mistimed command are candidate explanations. Design a test that helps distinguish them.
2. Check that the interfaces mean the same thing
A compatible robot name is a starting point. Compare the exact embodiment, tool and control interface. Record joint order, units, coordinate frames, gripper range and whether an action requests a position, velocity, effort or local control target. A command with the correct shape can still have the wrong physical meaning.
Check observations just as carefully: camera placement, intrinsics, image size, depth availability, robot state and timestamps. Keep capture time distinct from delivery time. A fresh inference response can contain an old frame.
MuJoCo exposes multiple actuator and contact-model choices. Inspect the model actually used for training before matching it to a physical controller; the simulator name alone does not describe those choices. MuJoCo modeling documentation.
3. Measure the gap that matters to your task
Use the following as a planning checklist. These measurements are useful only when the setup can actually provide them. A video frame is not a force sensor, and a motor command is not a measured contact force.
| Question | What to record | Decision it can inform |
|---|---|---|
| Does the control interface match? | Command units, applied target and measured robot state | Fix an action mapping before evaluating the policy |
| Does timing change behavior? | Capture, delivery, inference and command timestamps; gaps and stops | Change cadence, buffering or execution placement |
| Does the grasp hold? | Object motion, gripper feedback and calibrated force data if available | Investigate the grasp, tool or contact assumptions |
| Do the observations transfer? | Original camera frames and known scene variations | Adjust sensing or training data |
| Does the outcome repeat? | Every attempt, initial state, outcome and intervention | Estimate reliability within the tested conditions |
Collecting everything can make the experiment harder to interpret. Start with the smallest set of observations that distinguishes your leading explanations. Add instrumentation when it resolves an actual uncertainty.
4. Write the protocol before looking at the result
Freeze the controller or checkpoint, task definition and setup. Specify the starting conditions, permitted variation, success rule, trial count, time budget and stop conditions. Ask the hardware operator to review the permissible motion and recovery procedure before execution.
A first run can check connectivity, observation freshness and bounded control. A later run can evaluate the task. Keeping these questions separate makes an interface failure easier to diagnose without treating it as a policy result.
Choose trial counts based on the decision and available budget. One successful attempt is evidence that the behavior occurred once under those conditions. It is insufficient to establish broad reliability. Report the denominator, excluded attempts and uncertainty alongside any success percentage.
For a comparison, use a declared set of starting conditions and retain their order. Change one relevant factor where practical. Record resets and operator assistance; both can influence what a result means.
5. Keep a result another person can inspect
Your result package should connect the question to the inputs, execution and observations. Include the source or checkpoint identity, robot and tool configuration, calibration record, initial conditions, commands, original observations, timing and outcomes. Mark unavailable measurements as unavailable.
Preserve failed trials, stops and uncertain acknowledgements. Keep an agent’s interpretation separate from the measurements it cites. If a controller changes after a failure, give the next trial a new input identity rather than replacing the earlier result.
Use a held-out hardware test after tuning against earlier trials. Otherwise, it becomes difficult to tell whether a change improves transfer or only fits the examples already seen.
6. Give your coding agent a concrete experiment brief
An agent can help inspect interfaces, propose a protocol and analyze results. Give it an explicit question, evidence requirements and an owner-approved boundary. A useful brief could look like this:
Question: Does this controller retain the object during a slow transfer? Inputs: controller revision, robot/tool specification, object description Observations: original video, robot feedback, applied-command timing Protocol: declared starting states and an agreed set of repeated trials Outcome: retention and placement criteria defined before execution Limits: reviewed motion envelope, finite duration, owner-approved budget Report: every attempt, interventions, failures and remaining uncertainty
This is an illustrative brief, not an executable API request. Compatibility and local hardware limits must be established before anyone runs it. The result should tell you what to test or change next, even when the task fails.
Related field note: What SoftMimicGen’s cloth manipulation results do—and do not—establish.