Reproduce a study, then change one thing
P10 uses a foundation/regression branch. The dependency-free Python reference and browser fit the same seeded models on the same synthetic splits. It is a scaled-down study, not a frontier-paper reproduction. Use Python 3 and a terminal; no account, API, GPU or network inference is needed.
Download supplied reference Download learner exercise
- Run
python research-study.py --output baseline-study.json. The exclusive output write refuses to overwrite an existing artifact. - Export a completed browser study with the same recipe. Compare every task/seed, weights, scores and fitting visits; use absolute tolerance 10⁻¹⁰ for these bounded floating-point values.
- Implement the separate learner exercise: validate task/seed identities, compute paired task means and the factorial contrast. Supplied reference code is not your own implementation evidence.
- Propose one change. Freeze a versioned protocol, preserve the source hash, data formula, split IDs, seeds and all results. New claims need independently reserved tasks because the supplied cases are public.
- Write a negative or positive result-linked report, request an independent replay and record review status honestly. Public checks and self-review do not establish independent proficiency.
The original two-game notebook and baseline starter remain available as a different branch. Their programmed search contracts and reports are preserved; these regression results do not imply neural transfer between games.
Reproduce the original two-game branch
Download the unchanged baseline module and its manifest wrapper into one directory. Run node research-two-game-manifest.mjs my-reference.json development. The wrapper records the original source hash, selected split identities and hashes/counts for unselected partitions, budget, exact checks and all method/family results. Compare with the supplied development reference. Optional reserved/variants arguments explicitly expose those public cases; retain that exposure in your protocol. Methods are programmed BFS/greedy with or without visited-state memory, not trained recurrence or retrieval.
A complete small development comparison
Train on twelve synthetic rows per task. Development uses twelve different x values; all tasks and seeds here are public repeated practice. The optional quadratic feature adds capacity; weight shrinkage changes the fitting objective.
MAE: baseline 0.17957, candidate 0.08730. Improvement (baseline − candidate): 0.09227. Each pair uses 1920 fitting-example visits, excluding scoring and other overhead. Plot clips errors above 1; exact values remain here.
Inspect fitted weights and split membership
Weights: 0.211927, 0.229801, -0.344860.
Training: task0/row0, task0/row3, task0/row6, task0/row9, task0/row12, task0/row15, task0/row18, task0/row21, task0/row24, task0/row27, task0/row30, task0/row33.
Development: task0/row1, task0/row4, task0/row7, task0/row10, task0/row13, task0/row16, task0/row19, task0/row22, task0/row25, task0/row28, task0/row31, task0/row34.
Pair first, then summarize tasks
| Task | Seed | Baseline MAE | Candidate MAE | Difference |
|---|---|---|---|---|
| 0 | 7 | 0.1796 | 0.0873 | 0.0923 |
| 0 | 19 | 0.1796 | 0.0866 | 0.0930 |
| 0 | 41 | 0.1795 | 0.0863 | 0.0932 |
| 1 | 7 | 0.0249 | 0.0298 | -0.0049 |
| 1 | 19 | 0.0249 | 0.0293 | -0.0044 |
| 1 | 41 | 0.0249 | 0.0291 | -0.0041 |
| 2 | 7 | 0.1793 | 0.0636 | 0.1157 |
| 2 | 19 | 0.1793 | 0.0644 | 0.1149 |
| 2 | 41 | 0.1793 | 0.0647 | 0.1147 |
Task-cluster mean 0.06782. Exact 27-resample percentile illustration: [-0.00446, 0.11511]. Only three authored task clusters; this is not a reliable population guarantee. Seeds share task data and are averaged within each task.
Four-cell ablation
- A off, B off: error 0.17957
- A off, B on: error 0.17809
- A on, B off: error 0.08730
- A on, B on: error 0.10999
Interaction E(A+B) − E(A) − E(B) + E(base) = 0.02416. Negative means a larger error reduction together than the additive prediction. Equal updates do not imply equal arithmetic cost.
Continue to the frozen research projectSave the complete research project
Authorship, synthetic-data provenance and distribution notice