Observe the task before proposing a feature
How do recruitment and question choice change the evidence?
Start with a concrete question
Ten fictional participants differ in experience. Task failure rates are supplied teaching assumptions, not measured interview outcomes.
Try the initial inputs, then write a prediction before changing one setting. All calculations run locally on your CPU.
What do you expect?
Before changing a setting: How do recruitment and question choice change the evidence? Write what you expect and why.
Change one thing. Inspect what happens.
- Compare convenient recruitment with a balanced group.
- Read the two prompt types and name what each can establish.
- Calculate expected task failures under each mix.
- Write a problem statement naming user, task, observed barrier and missing evidence.
Your settingsMechanism
Calculation belowEvidence
Compare outcomes
Compare the current results
Every bar starts at zero and uses the same numeric scale in this chart. The upper bound automatically adjusts to the largest current value (at least 1), so compare the numbers when changing inputs. Units and quantities come from each label; different kinds of quantity are not interchangeable. Displayed values are rounded to at most six significant digits.Follow the calculation
- Recruitment: 8 frequent and 2 occasional users.
- Fixture task failure rates: frequent 25%, occasional 75%.
- Prompt: “Would a better dashboard be helpful?” This solicits an opinion, not a task attempt.
- 3.5 expected failures applies to observed tasks, not stated preferences.
Start with the user task and recruit beyond convenient participants. A favorable opinion is not evidence that the task succeeds.
Write an evidence-based problem statement
Name a user group, task, barrier and supporting observation. Separate observed behavior from preferences and list one missing fact.
71/8,000 characters. Drafts save locally and are included in backups. Importing or opening a draft never runs it.
Compare with my earlier draft
Users want a better dashboard, so we should build lots of new features.
Explain what you changed and why in your observation notes. Text differences do not establish quality.
Inspect an example solution after your attempt
Occasional users in this fictional task model have more navigation failures than frequent users. Investigate how occasional learners find saved work with a task-based study; a positive opinion about dashboards does not identify the navigation barrier.
This is one worked example. Test another input and explain its limitations.
Captures keep your inputs, prediction and observation. Results are recalculated from those inputs. Resetting inputs keeps your captures and challenge history.
Explain the mechanism
Observe the task before proposing a feature
Ten fictional participants differ in experience. Task failure rates are supplied teaching assumptions, not measured interview outcomes.
Follow the mechanism
Experienced participants can hide problems faced by occasional users.
A stated preference does not directly establish whether a task can be completed.
Use the fixture calculation only for task attempts; an opinion question does not generate those observations.
Worked example
Recruitment: 8 frequent and 2 occasional users.
Fixture task failure rates: frequent 25%, occasional 75%.
Prompt: “Would a better dashboard be helpful?” This solicits an opinion, not a task attempt.
3.5 expected failures applies to observed tasks, not stated preferences.
Calculation
Expected task failures = frequent users × 0.25 + occasional users × 0.75.
Try a controlled comparison
Compare convenient recruitment with a balanced group.
Read the two prompt types and name what each can establish.
Calculate expected task failures under each mix.
Write a problem statement naming user, task, observed barrier and missing evidence.
What this does not establish
- All messages, people, documents and outcomes here are original fictional teaching fixtures; results are not measurements of real organizations.
Try a fresh case
Four frequent users fail 25% of attempts and six occasional users fail 75%. What is the expected total number of failures?
This is a public practice check. Retries support learning; matching a number does not certify independent proficiency.
Take the investigation further
Choose any extension. These are project prompts for you to carry out; they are not automatically assessed.
School investigation
Create a new case with different numbers and explain your decision to a classmate.
College investigation
Implement or analyze a fresh case, document assumptions and compare at least two alternatives.
Workplace investigation
Prepare a decision artifact with evidence, operating constraints, an owner and a follow-up measurement.
Keep the evidence, choose your next step
Completion records your own decision that you have worked through this investigation. Practice checks and independent project evidence are separate.