Repair data without leaking answers.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

The score answers the split’s question

A nearest-neighbor model copies the label of the closest training row. A duplicate in validation can therefore make the task almost trivial. That measures recognizing a repeated measurement, not predicting a new flower.

The local Iris source has 150 examples, four measurements in centimetres and three species. Our teaching view deliberately creates duplicates, missing petal lengths and some sepal lengths recorded in millimetres. Inspect the original view before attributing those issues to the source. The manifest records its license, hashes and transformations.

Choose the split to match the intended use. For a new person or site, keep that identity in one split. For future events at existing sites, hold out later times. For both new sites and future times, both boundaries matter. The Iris groups here use identical measurements; they are not verified specimen identities.

Repair one measurement, then fit the transformation

Suppose a recorded length is 60 mm. Divide by 10 to get 6 cm. If the training mean is 5 cm and its standard deviation is 0.5 cm, the standardized value is 2: two standard deviations above the training mean.

Here x is the converted measurement, μ is the training average, s is the training standard deviation (a measure of spread), and z is a unitless coordinate. Treating 60 mm as 60 cm would produce 110 instead of 2 and distort which flower appears nearby.

Fit the mean using observed training cells. Fill missing cells with that mean, then calculate the standard deviation of the filled training column. Apply those same fitted values to validation. An entirely missing feature has no defined mean; stop rather than invent a number. A constant feature uses scale 1 to avoid dividing by zero.

Why the nearest flower wins

For each of four standardized measurements, subtract the candidate training value from the validation value. Square each difference, add them, then take a square root. The smallest distance wins; ties use the first training row. Labels do not enter these distances.

a and b are two rows, j identifies a measurement, and d is their distance. For differences 1, 0, 2, 0, squared distance is 5 and distance is about 2.236. Compare the model with a majority-label baseline: a complicated procedure must justify its added value.

Report a limited claim

A clean pipeline can score lower because it asks a harder question. Changing splits also changes the training and validation examples, so the difference is not a controlled estimate of one repair’s effect. Repeatedly inspecting validation makes it development evidence. It is not an untouched final test or a guarantee about future populations.

For the project, capture the default run and a grouped, training-only, unit-corrected run. State counts, baseline, repairs and a limitation. Then attempt the sensor-site and future-period assessment cases without consulting the worked arithmetic. The project records first submissions separately from practice retries.