Repair a messy event log
Which rows are observations and which are recording errors?
Entering midway? Revisit Who answered the survey? for the preceding worked example, or start here and use the explanation below.
Start with a concrete question
Six original synthetic service records contain one repeated event ID, one missing duration and one impossible negative duration. The trace shows every row.
Try the initial inputs, then write a prediction before changing one setting. All calculations run locally on your CPU.
What do you expect?
Predict whether replacing missing with zero makes service appear faster.
Change one thing. Inspect what happens.
- Disable all repairs and inspect each source row.
- Enable duplicate and validity checks separately; name the contract each enforces.
- Compare excluding missing with inserting zero. Write a data-quality note before quoting the mean.
Your settingsMechanism
Calculation belowEvidence
Compare outcomes
Compare the current results
Every bar starts at zero and uses the same numeric scale in this chart. The upper bound automatically adjusts to the largest current value (at least 1), so compare the numbers when changing inputs. Units and quantities come from each label; different kinds of quantity are not interchangeable. Displayed values are rounded to at most six significant digits.Follow the calculation
- a: keep 10 minutes.
- b: keep 20 minutes.
- c: keep 30 minutes.
- c: duplicate event excluded.
- d: missing duration excluded from mean, retained as missing in audit.
- e: negative duration quarantined.
- Retained sum 60 / 3 rows = 20 minutes.
Repair according to a documented contract. Missing is not zero, and duplicate records are not independent new events.
Repair an editable event table
Keep a copy of the original rows in your notes. Remove the repeated event c, quarantine the negative duration, and decide how to represent missing d without inventing zero. Inspect the table after editing and explain every deletion. The inspector validates numeric cells, not event identity or the wisdom of a repair.
44/8,000 characters. Drafts save locally and are included in backups. Importing or opening a draft never runs it.
Compare with my earlier draft
event_id,minutes a,10 b,20 c,30 c,30 d, e,-5
Explain what you changed and why in your observation notes. Text differences do not establish quality.
Inspect an example solution after your attempt
event_id,minutes a,10 b,20 c,30
This is one worked example. Test another input and explain its limitations.
Captures keep your inputs, prediction and observation. Results are recalculated from those inputs. Resetting inputs keeps your captures and challenge history.
Explain the mechanism
Repair a messy event log
Six original synthetic service records contain one repeated event ID, one missing duration and one impossible negative duration. The trace shows every row.
Predict before changing inputs
Predict whether replacing missing with zero makes service appear faster.
Understand the mechanism
An observation is an event, not necessarily a row. Repeated ID c represents the same event and must not gain extra weight.
With duplicate and negative checks enabled and missing excluded, valid values are 10, 20, 30: mean 60/3 = 20 minutes. Zero-filling the missing record gives 60/4 = 15.
Quarantine means retain a rejected record for investigation rather than silently rewriting it. This fixed fixture remains visible in the trace.
Calculate it yourself
Mean = retained sum / retained row count. Valid duration ≥ 0; one row per event ID.
Guided investigation
Disable all repairs and inspect each source row.
Enable duplicate and validity checks separately; name the contract each enforces.
Compare excluding missing with inserting zero. Write a data-quality note before quoting the mean.
Assumptions and limitations
Event identities and the meaning of negative durations are known here. A real repair must consult the data contract.
Excluding missing rows can still bias an estimate. No repair creates the absent measurement.
Further investigations
High school: Audit an invented school activity log and list exclusions.
College: Implement a validator retaining raw rows, reason codes and a clean view.
Working professionals: Prepare a dashboard data-quality specification with missingness counts and owners.
Your implementation or data task
Keep a copy of the original rows in your notes. Remove the repeated event c, quarantine the negative duration, and decide how to represent missing d without inventing zero. Inspect the table after editing and explain every deletion. The inspector validates numeric cells, not event identity or the wisdom of a repair.
Use the editable workspace on this page. Its output is practice evidence; explain why it satisfies the contract and test a fresh case.
Optional public-data follow-up
The existing local Iris CSV contains 150 plant-measurement rows from Fisher (1936), distributed by UCI under CC BY 4.0. Read the supplied attribution before reusing it. Its row IDs were generated for this site; they are not biological specimen IDs. The existing data-quality lesson introduces synthetic teaching alterations in memory and labels them as alterations, not original source defects.
Download local Iris CSV · Attribution and alteration notes · Work through its data-quality investigation
Try a fresh case
New valid records are 5, 15, 25, with one missing row. If you exclude missing, what is the mean?
This is a public practice check. Retries support learning; matching a number does not certify independent proficiency.
Take the investigation further
Choose any extension. These are project prompts for you to carry out; they are not automatically assessed.
School investigation
Audit an invented school activity log and list exclusions.
College investigation
Implement a validator retaining raw rows, reason codes and a clean view.
Workplace investigation
Prepare a dashboard data-quality specification with missingness counts and owners.
Keep the evidence, choose your next step
Completion records your own decision that you have worked through this investigation. Practice checks and independent project evidence are separate.