P09 · Course 13 bridge

Implement a policy and challenge its embodiment

First reproduce the browser's 110-parameter imitation mechanism in standard-library Python. Then use the pinned MuJoCo arm starter to compare a trained torque policy with a feedback baseline. The second task has a different observation/action contract: it is a richer-embodiment comparison, not direct weight transfer.

No-install preview

1. Observation: object (−0.5,0) m; goal (+0.75,0) m; instruction right = [1,0].
2. Network: six inputs → eight tanh units → six tanh outputs.
3. One output command (+1,0) m/s for 0.1 s at drag 0.2 changes x by 0.08 m.
4. Execution receipt: object (−0.42,0) m. Check bounds and task outcome independently.
5. Torque bridge: a goal error 0.5 rad with velocity 1 rad/s requests 8×0.5−3×1 = 1 N m from the baseline. A velocity number cannot replace this calculation.

1. Reproduce, implement, repair

Download the planar Python reference and learner exercise. Python 3.12 or later; no dependencies. Reference checks compare all 110 gradients to finite differences and a separately derived zero-network oracle. The untouched learner mode intentionally raises NotImplementedError.

python robot-learning.py --check
python robot-learning.py --updates 100 --output reference-100.json
python robot-learning.py --mode learner --check
python robot-learning.py --mode learner --updates 100 --output learner-100.json

Complete learner_gradient without calling the reference gradient. Explain the loss denominator and same-checkpoint update. Deliberately remove a contribution or update a weight early, show which test detects it, then repair the fault. Files use exclusive creation so earlier results are retained. Add your own test on different values before asking a reviewer for an unfamiliar case.

2. Use a standard simulator

Download the pinned CPU MuJoCo starter. Read its README and licence. The archive includes a two-joint arm with torque dynamics, independent sampled geometry/limit checks, a 114-parameter imitation model, reference tests, and separate learner TODOs. The environment pins MuJoCo 3.3.7 and NumPy 2.3.5. Package download needs internet; execution is local and headless.

python -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt
.venv/Scripts/python checks.py
.venv/Scripts/python run.py train --seed 7 --output policy-7.json
.venv/Scripts/python run.py evaluate --model policy-7.json --output results-7.json

Windows commands shown; macOS/Linux use .venv/bin/python. Repeat seeds 19 and 41 with distinct filenames. Preserve every failure and time limit. Do not report a missing run as zero success. Source hashes and numerical replay identify the recipe, not trusted authorship.

3. Test transfer honestly

Before the final run, write a new scenario specification: initial angles, target, damping, noise and obstacle. Freeze code and weights; have an independent reviewer choose unfamiliar values within the declared contract. Public starter scenarios are development evidence. Check the delivered observation/action boundary, invalid commands, sampled collisions and termination. Never give simulator truth to the controller.

Compare planar metres/second with joint torque, exact rendered position with noisy joint sensors, permanent contact with rigid-body dynamics, and 0.1 s steps with 0.005 s integration. Neither environment validates a physical robot. The MuJoCo checks sample every 5 ms and do not prove clearance between samples.

Deliverables

Save source/environment, demonstration provenance, gradient tests, frozen model, all three seed results, complete receipts, full-task denominators and an embodiment comparison. Attach selected code or file references inertly to Project 13. Guided completion does not require this studio; implementation proficiency requires runnable artifacts and independent review. Human review and learner pilots remain pending.

Authorship, synthetic-data provenance and distribution notice