MuJoCo physics simulation workshop — procedural guidance for building and tuning a PID controller for ball-on-plate balancing with a Franka Panda robot arm. Activates when users build simulations, write PID controllers, explore joints, stream live video, or troubleshoot ball-balancing tasks.
Procedural guidance for the ball-on-plate balancing workshop. See CLAUDE.md for reference data (home pose, model paths, API).
Every script follows three phases:
MjModel.from_xml_path()data.ctrl[7] = 0.008mj_forward() to compute positionsLiveStreamer() (reads STREAM_PORT env var automatically)for each timestep:
mj_step()
hold non-PID joints at home pose
compute ball error = xpos[ball] - xpos[plate]
compute PID correction from error (P and optionally D terms)
apply correction: ctrl[joint_index] = home[joint_index] + correction
render and push frame to streamer
abs(error_x) > 0.14 or abs(error_y) > 0.14 or ball Z drops below plateSurvival Time: X.X seconds to terminalWrite a "just observe" script — load the model, hold the home pose, stream live. No controller. Watch the ball fall off naturally.
Single-joint isolation procedure:
mj_forward() and observe how the plate movesFollow this iterative process. Do NOT run diagnostics as the first step — write a PID attempt first and let it fail naturally.
mj_forward(), measure how much the plate position changes| Symptom | Diagnostic Step |
|---|---|
| Ball flies off immediately (<0.5s) | Check the correction sign — it may be pushing the ball off instead of correcting |
| Ball drifts slowly off one edge | Check if you are controlling the correct axis for that direction |
| Ball oscillates wildly on the plate | Gains may be too high — try reducing Kp significantly |
| NaN errors in simulation | Correction values may be exploding — add output clamping or reduce gains |
| Survival time stuck at ~1s regardless of gains | You may be controlling the wrong joints entirely — run the nudge diagnostic |
| "Port already in use" error | A previous script is still running — press Ctrl+C in that terminal first |
Run scripts/03_optimize_pid.py and observe the working PID controller. Compare its behavior with the failing baseline from Sprint 2. Explain what changed: correct joints, correct sign. This is observation only — no code editing.
Key discussion points:
03_optimize_pid.py control vs 02_pid_baseline.py?Guided first pass through the measurement and improvement loop:
scripts/04_survival_map.py — see the baseline Controller Score (~3.3 sec)05_challenge.pyscripts/04_survival_map.py --controller scripts/05_challenge.py — see the score changeWhen a participant asks in natural language to "test the modified controller with the survival map" or similar, translate to the command:
python scripts/04_survival_map.py --controller scripts/05_challenge.py
This teaches the full iteration workflow before Sprint 5's autonomous exploration.
Participants edit scripts/05_challenge.py (the controller playground) to improve the controller, then evaluate with scripts/04_survival_map.py.
05_challenge.pymake_controller() function inside 05_challenge.pypython scripts/04_survival_map.py --controller scripts/05_challenge.pyIn Sprint 5, Claude self-drives this loop: form a hypothesis, edit 05_challenge.py, run the survival map, compare scores, and iterate. The participant guides Claude with high-level goals in natural language.
The survival map prints a Controller Score = mean survival time in seconds across all grid positions. Baseline PID (Kp=2, Kd=0) scores ~3.3 sec. Higher is better. The score also appears on the contour plot.
05_challenge.py exports make_controller(model, dt, home):
def make_controller(model, dt, home):
"""Called once per trial. Return a controller function."""
def controller(data, plate_id, ball_id, step, t):
# Compute and apply corrections to data.ctrl
pass
return controller
Evaluate with: python scripts/04_survival_map.py --controller scripts/05_challenge.py
Always stream the result to the browser — avoid --no-stream.
05_challenge.py — do NOT modify 04_survival_map.pycam = streamer.make_free_camera(model) after streamer.start()streamer.drain_camera_commands(model, cam, renderer.scene) before renderer.update_scene(data, camera=cam) — this enables browser-based orbit/zoom/panrender_every = int(1.0 / (fps * dt)) to limit rendering to ~30fpstry/finally with streamer.stop() in the finally blockrenderer.render() from a thread other than the simulation thread (OpenGL is not thread-safe)LiveStreamer() with no port argument reads STREAM_PORT env var automatically — do not hardcode port numbers