Different mistakes have different costs.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

Start with the decision, not the percentage

An equipment desk can review only some requests urgently. A score ranks requests; a threshold is the cutoff for action. In this lesson, “positive” means urgent. It does not mean good, correct, or desirable. A true label says what actually happened, while a prediction is the system’s decision.

At threshold 0.50, a request scoring 0.60 receives urgent review and one scoring 0.35 does not. Lowering the threshold to 0.35 includes both. The model has not learned anything new: its scores are unchanged. Our supplied scores are invented rankings, not calibrated probabilities. A score of 0.80 does not establish an 80% chance of urgency.

Four boxes reveal the mistakes

The live matrix crosses the actual outcome with the predicted outcome. “True” means correct; “false” means incorrect. Positive and negative refer to the prediction.

These are the worked-example defaults, with 20 requests and threshold 0.50. The lab’s matrix and calculations change with your controls. There are four truly urgent requests (TP + FN) and six alerts (TP + FP). Count those groups before dividing.

Three rates answer three questions

Accuracy asks how many decisions were right overall: 16 out of 20, or 80%. Precision asks how many alerts were right: 3 out of 6, or 50%. Recall asks how many truly urgent requests were caught: 3 out of 4, or 75%.

Each symbol is a count from the table. The fractions are unitless rates from 0 to 1; multiply by 100 to express percentages. Read precision as “correct alerts divided by all alerts,” and recall as “caught urgent requests divided by all urgent requests.” They exchange the denominator, not the numerator.

With no alerts, precision has denominator zero and is undefined. It is not 100% and not evidence of a reliable detector. With no actual positives, recall is undefined. An empty sample has no accuracy. The lab labels these cases rather than silently inserting a favorable number.

Accuracy can reward doing nothing useful

The majority baseline always predicts the most common class. On the default cohort it predicts ordinary and achieves 16 / 20 = 80% accuracy, exactly matching the default score rule. But its recall is zero: it misses all four urgent requests. The score rule catches three. This is why class balance—the relative numbers in each class—and the intended use matter.

Changing the displayed mix removes twelve ordinary requests while retaining the four urgent ones. This illustrates sensitivity to the sample. It also changes the ordinary-score distribution, so it is not an isolated population-prevalence experiment. A small teaching cohort does not establish future performance or subgroup fairness.

Turn assumptions into an explicit cost

Suppose each unnecessary review costs 2 points and each missed urgent request costs 10 points. The default rule costs 3 × 2 + 1 × 10 = 16 points. Predicting ordinary for everyone costs 4 × 10 = 40. Costs express our teaching assumptions about consequences; they are not measured money or universal values.

C is total cost in points, c with subscript FP is cost per false alarm, and c with subscript FN is cost per miss. This model assumes constant costs per error, zero cost for correct decisions, and unlimited review capacity. Real decisions may need different costs for different cases, a review-capacity constraint, and human escalation.

Lowering the default threshold to 0.35 catches all four urgent requests but also creates five false alarms. At the original costs that is 10 points; if false alarms cost 10 and misses cost 2, it is 50 points. A threshold preference depends on the use and costs, not on a universally best cutoff of 0.50. Compare thresholds on the same cohort, since total cost also grows with the number of requests.

Separate practice from a release decision

Changing thresholds after inspecting outcomes is development. Choose the metric and costs, select a threshold on development evidence, freeze it, and then evaluate on fresh cases collected for the intended use. Reusing this tiny cohort cannot create new independent evidence. Examine affected groups, review workload, uncertain labels and future changes before recommending deployment.

Use the lab’s separate library calculation to practice the denominators. For a decision brief, record two compared thresholds, the counts and costs, your assumptions, and the next evidence you need. For implementation practice, build and test the count-and-rate function yourself, including the zero-denominator and exact-threshold cases. Neither a downloaded brief nor passing the lesson’s practice question certifies independent implementation or decision quality.