Learn the pattern, not the examples.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

Three datasets, three jobs

Training data fits the parameters. Validation data helps you choose settings such as model size or regularization. Test data estimates the final chosen model’s performance after those choices are frozen. Reusing the test score to make choices turns it into another validation score.

The lab uses 12 training points, 24 validation points, and 24 separately generated test points. All come from the same smooth toy relationship with independent noise. Circles are training observations; diamonds are validation observations. The curve is fitted only to the circles. The final test observations remain hidden until you freeze your choice.

A split must match the real task. If two almost identical photos of the same object land in different sets, the test may be too easy. For predicting the future, a time-based split can be more appropriate than shuffling everything. No split repairs unrepresentative or incorrectly labeled data by itself.

The most flexible curve is not always the best predictor

A polynomial adds powers of the input. Degree 1 is a straight line; a higher degree can bend more. In this lab, degree 11 has enough flexibility to pass through all 12 training points. Some of those points contain random noise, so a perfect fit can make the curve worse between them. This is overfitting. A model too restricted to capture the useful pattern underfits.

Compare degrees 1, 4, and 11. Training error falls as flexibility increases, but validation error does not keep falling. The comparison table reports actual measured errors from the fits; it is not an invented performance curve. Here, fitting solves a small least-squares system directly rather than running the gradient-descent animation from the previous module.

Every learning process has preferences

An inductive bias is a preference among explanations that could fit the observations. Model architecture, initialization, optimizer, regularization, and data all shape that preference. A model looking at neighboring pixels favors different patterns from one comparing words across a sentence.

Regularization adds a cost for certain parameter choices. Our lab penalizes the squared nonconstant polynomial coefficients. The fit therefore trades training error against that penalty. The displayed losses remain ordinary mean squared errors, without adding the penalty. This particular penalty depends on the numerical basis used to express the polynomial; it is one teaching choice, not a universal measure of simplicity.

More parameters do not automatically imply worse generalization. Modern networks can generalize even when they have enough parameters to memorize their data. Parameter count alone does not describe the solution actually found.

A good test has a boundary

Even an independent test only samples the conditions it contains. A model trained on bright photographs may fail in dim light. This is distribution shift: the conditions change. Test meaningful variations and report failures alongside successes.

Our frozen test is a small estimate, not proof of real-world reliability. If you reopen tuning after seeing it, it is no longer an unseen test. The lab keeps that warning even after a reset. In a real project, reserve new independent evidence for the next final claim.