Learn

Learning preferences

Make learning more comfortable. Changes apply immediately.

Enlarges page content, including diagrams and their labels. You can also use your browser’s zoom.

Choose the colours that are easiest for you to distinguish. Applies to lesson diagrams and lab plots. Labels, shapes, hatching and line patterns also distinguish meanings. Source photographs and raw image brightness stay unchanged; mask-only views are available for image overlays.

ExampleComparisonSample link

Saved in this browser on this device. Preferences are separate from progress backups.

Lesson 9 of 9 in this unit

Build AI Systems · ORIGINAL LEARN AI LESSON

Update without silently changing the contract.

Train a real candidate, recompute old/new regression gates and retain both checkpoints for a replayable rollback record.

By the end: Separate optimization, candidate selection, promotion criteria and rollback provenance while retaining failed comparisons.

Start with the exampleJump to the experimentRead the explanation
Prerequisites and path options

Why learn this? Preserve the trained baseline, regression gates and rollback record when deciding whether a real candidate should replace a working model.

01 · TRY

Start with something concrete.

A new calibration model improves new devices while damaging older ones. The largest improvement is not permission to change an agreed retention requirement.

Try the interactive model
02 · PREDICT
Before you experiment

A candidate improves new MSE but exceeds the predeclared old-task regression tolerance. What should the gate do?

A candidate improves new MSE but exceeds the predeclared old-task regression tolerance. What should the gate do?
03 · EXPERIMENT

Change one thing. Watch what follows.

Loading this browser experiment…

04 · UNDERSTAND
READ & CONNECT

Understand what changed.

Back to the experiment

A shorter explanation, with the original worked examples available below. Both explain the same experiment and its limits.

Same learning goal in both modes: Separate optimization, candidate selection, promotion criteria and rollback provenance while retaining failed comparisons.

The idea in everyday terms

A new model can improve one device family while damaging another. Decide what must be retained before selecting a candidate. This bench trains real weights, preserves the old checkpoint and checks both old and new monitoring errors.

What to watch in the experiment

The gate allows old-task MSE to rise by at most 0.02 and requires new-task MSE to fall by at least 0.01. Both conditions must pass. Compare new-only fitting and replay from the same baseline and update budget. Replay the result from the stored parameter arrays rather than trusting imported score numbers.

What this does not establish

Rollback restores the old active model while retaining the candidate and its receipt for audit. Repeatedly viewed monitoring cases remain development evidence after freezing. A saved numerical record cannot authenticate how imported weights were trained, and report presence cannot judge reasoning quality. The project adds separate transfer cases; independent implementation review remains pending.

Open the full worked examples, calculations and limitations

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

Improvement is a joint requirement

A model trained for a newer device can become worse on an older one. If the product promised to retain the older capability, a large new-task improvement is not permission to ignore that regression. A release decision needs a declared baseline, candidate, measurements and rule before looking at the preferred outcome.

This bench trains an actual thirteen-parameter tanh network. First train the old-task baseline for 100 updates and freeze its parameters. Then adapt a candidate using new-only or replay fitting. All updates use training coordinates. Separate monitoring grids measure old and new mean squared error and never enter gradients.

Calculate the gate

The versioned rule adaptation-release-v1 allows at most 0.02 old-task MSE regression and requires at least 0.01 new-task MSE improvement. Both conditions must hold. These teaching thresholds are fixed for this bench; they are not calibrated deployment safety standards.

Suppose baseline errors are 0.10 on old tasks and 0.80 on new tasks. A candidate produces 0.115 and 0.50. Its old regression is 0.015, below 0.02, and its new improvement is 0.30, above 0.01. Both pass. Another candidate with new MSE 0.20 but old MSE 0.14 fails because old regression 0.04 exceeds the protected limit.

The gate recomputes predictions from both saved parameter arrays. Imported scalar scores are not trusted. An eligible result records the exact checkpoints, suite version and thresholds, and a failed result retains its baseline instead. A frozen selection records a decision; it is distinct from automatic promotion.

Rollback preserves evidence

Rolling back makes the retained baseline the active model while keeping the candidate and release receipt available. Deleting the failed run would hide information needed to understand the next change. Compare the active-model label with the stored candidate; those objects answer different questions.

Changing replay strategy restores the original baseline and resets candidate updates. This makes a same-start comparison possible. Match update count, buffer capacity and frequency setting, and report the different new-versus-retained composition of fitting work. The no-adaptation control provides an explicit “do nothing” alternative.

What the record proves and what it does not

These monitoring outcomes are public and repeatedly visible. A gate built from them is development evidence, even after the candidate is frozen. A separate future evaluation needs a predeclared suite whose exposure is tracked; a new filename does not make old cases fresh.

The browser validates numerical bounds and recomputes metrics. It cannot authenticate that imported parameters were produced by the stated historical training sequence. Report that distinction alongside provenance. A text-presence check also cannot judge the quality of a decision brief or independently run learner code.

The cumulative adaptation project saves no-adaptation, new-only and replay checkpoints, asks for a retention/cost decision, and supplies separate objective transfer cases. Its implementation studio requires reproducible local outputs and all-run reporting. Independent human review and representative learner testing remain separate work.

05 · APPLY
Check your understanding

The public monitoring cases were repeatedly used to select a candidate. What does freezing it establish?

The public monitoring cases were repeatedly used to select a candidate. What does freezing it establish?
06 · CONTINUE

Take this idea with you.

A release receipt binds a specific baseline, candidate, suite and thresholds. Rollback restores behavior while preserving the record of why the update was tried.

My Learning
Explore related topics and the topic map

How this topic connects

Explore this topic on the interactive map

Use these connections to review a foundation or choose where to go next. You can open any topic without completing the others.

What it builds on

  • Controlled improvement

    Require evidence before promoting a change and keep a way to reverse harmful regressions.

  • Replay selection

    Choose retained experience using coverage and budget evidence instead of assuming that a larger replay count means more independent information.

You are hereUpdate without silently changing the contract

Train a real candidate, recompute old/new regression gates and retain both checkpoints for a replayable rollback record.

What it enables

  • Adaptation project Apply in a project

    Carry this comparison into the saved adaptation project, with matched baseline checkpoints, retained failures and separate transfer evidence.