Lesson 2 of 4 in this unit
Answer with evidence.
Build an extractive answer with precise citations, reject unsupported claims, and make conflicts and missing facts visible.
By the end: Separate relevance, publisher eligibility, freshness, exact claim support and contradiction when deciding what to answer or leave unresolved.
Prerequisites and path options
Why learn this? Check whether citations support each claim so a convincing answer does not hide missing or conflicting evidence.
Start with something concrete.
Two current office notices give different library closing times. Both are easy to quote. Can either quotation establish the current closing time? Inspect both sources before assembling an answer.
Try the interactive modelA citation points to an official notice. Does that make every claim in the answer supported?
Change one thing. Watch what follows.
Loading this browser experiment…
Understand what changed.
A shorter explanation, with the original worked examples available below. Both explain the same experiment and its limits.
The idea in everyday terms
A citation shows where a claim came from, but the passage must actually support the requested fact. Two current sources can conflict even when both are quoted exactly. Repeating one citation cannot resolve a contradiction or fill a missing fact.
What to watch in the experiment
Check relevance, freshness and source policy separately from exact quotation. Include only supported facts in the answer, and make missing or disputed facts visible. A partial answer can be useful without pretending the whole request is resolved.
What this does not establish
The lab uses authored fact labels and exact spans, not learned understanding or a truth detector. Source support also does not authorize external actions.
Open the full worked examples, calculations and limitations
This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.
A citation is a place to look
A school office notice says the library closes at 17:00 on weekdays. A second current office notice says 18:00. Both can be quoted exactly. Neither quotation resolves the disagreement. Before giving a time, an assistant needs clarification from the source owner or an explicit rule establishing which notice supersedes the other.
The lab supplies fictional sources with visible publisher, freshness and fact labels. You select a proposed claim and the exact sentence that should support it. The final answer extracts only supported source text and links back to that span. This is an extractive answer: it copies the relevant text rather than generating a new explanation.
Follow a claim through separate checks
| Check | Question to ask | Counterexample |
|---|---|---|
| Availability | Is this source and span actually in the available collection? | A removed notice cannot support the current answer |
| Relevance | Does this claim address the user’s requested fact? | A weekday fact does not answer a holiday question |
| Freshness | Does this source apply to the stated time or version? | Last term’s correct policy may be wrong this term |
| Publisher eligibility | Is this publisher suitable under the declared evidence policy? | An unverified discussion post is insufficient for an official schedule |
| Claim support | Does the selected span state the proposed fact? | The fee sentence does not establish closing time |
| Conflict | Does another eligible current source disagree? | Two current office notices state different times |
These checks are not interchangeable. A source may be authoritative for one subject and unsuitable for another. An official publisher can still make mistakes. A fresh source can be irrelevant. A passage may precisely state something without being reliable enough to use.
The lab’s eligibility and current flags are authored scenario assumptions, not scores inferred from the text. Its exact-support rule compares explicit fact labels; it is not a general language-entailment model. An actual application needs maintained source provenance, versions, dates and a policy for disputed evidence.
Support every requested part
The library question asks for two facts: weekday closing time and whether borrowing is free. Selecting both matching spans covers 2 ÷ 2 = 100% of these declared facts. Selecting only the fee sentence covers 1 ÷ 2 = 50%, so the answer is partial. Citing the same closing-time sentence twice still covers only one fact.
This fraction measures coverage under the exercise’s rules. It is not the probability that the answer is true. More citations do not automatically mean more distinct evidence, especially if sources copied one another.
The answer panel keeps rejected proposals separate from the extracts it actually includes. Even when all requested facts have supported extracts, an additional rejected proposal remains visible in the audit. Do not report that every proposed claim was correct merely because the final filtered answer covers the question.
Handle disagreement without hiding it
Add the correct closing-time and fee claims, then introduce the conflicting office notice. The fee extract can remain; the closing-time claim becomes unresolved. The checker inspects all eligible current sources in the available collection, not only the citation you chose. This prevents selecting a convenient quotation while ignoring a known contradiction.
Absence of a conflict in this small collection is not proof that no conflict exists elsewhere. Retrieval may have missed a relevant source. Neither choosing the first result nor averaging 17:00 and 18:00 establishes the policy. A valid resolution might use a verified superseding notice, but this scenario deliberately supplies no such relationship.
Practice: what survives a conflict?
If a new notice disputes only the closing time, the fee fact remains supported by the current office fee sentence. Include that extract and explicitly state that the closing time is unresolved. If the disputed fact is the only requested fact, abstain from a factual answer until the disagreement is resolved.
Make abstention informative
The holiday question has no matching source in this collection. A weekday quotation is insufficient even if it comes from the right publisher. The answer should state that holiday hours are unavailable, not silently reuse the weekday time.
The lab distinguishes supported, partial and abstain based on the supported requested facts you have actually selected. “Abstain” with no claims selected means you have not supplied evidence; it does not prove that no source could answer. Inspect the available collection and the reported failed checks before explaining the gap.
Removing the current office notices invalidates citations to them. The archived notice and discussion post remain visible, but cannot meet the current official-policy requirement. Making a source unavailable is not the same as proving it false.
Apply the evidence policy
For guided practice, export a normal answer and compare it with the same claims under conflicting sources. Explain which checks change. For implementation practice, write tests for a valid source with an incorrect span, a removed source, a duplicate citation and an irrelevant but accurately quoted claim. Include contradictory eligible sources that the answer did not cite.
For a management decision brief, specify who maintains the source collection, which facts require human resolution, and how a partial answer should be presented. An evidence policy that always refuses may avoid some wrong answers while failing to help users; report useful supported coverage as well as rejected claims and unresolved requests.
These are public practice cases, not independently reviewed project evidence. The broader evidence-assistant project will connect retrieval, answer construction, tool boundaries and evaluation. Source support alone does not authorize an external action.
Two eligible current sources conflict about closing time but agree that borrowing is free. What should an answer do?
Take this idea with you.
A citation identifies evidence to inspect. Answer only the supported parts and expose missing, outdated or contradictory evidence.
Explore related topics and the topic map
How this topic connects
Explore this topic on the interactive mapUse these connections to review a foundation or choose where to go next. You can open any topic without completing the others.
What it builds on
- Find relevant information
Find useful source passages for an assistant while spotting when a high search score still leaves the question unanswered.
- Define an assistant’s job
Specify what an assistant may answer or do before giving it access to tools and documents.
Build an extractive answer with precise citations, reject unsupported claims, and make conflicts and missing facts visible.
What it enables
- Use a pretrained model responsibly
Decide whether an existing model fits a task before trusting its fluent output or adapting it.
- Keep documents from becoming instructions
Prevent retrieved text from granting itself permission to perform actions.
- Measure the whole workflow
Find failures across retrieval, tools and answers that a single component score would miss.