Learn

Learning preferences

Make learning more comfortable. Changes apply immediately.

Enlarges page content, including diagrams and their labels. You can also use your browser’s zoom.

Choose the colours that are easiest for you to distinguish. Applies to lesson diagrams and lab plots. Labels, shapes, hatching and line patterns also distinguish meanings. Source photographs and raw image brightness stay unchanged; mask-only views are available for image overlays.

ExampleComparisonSample link

Saved in this browser on this device. Preferences are separate from progress backups.

Lesson 2 of 6 in this unit

Build AI Systems · SOURCE LESSON 20

Call a tool, then check the evidence.

Propose structured calls, inspect independent runtime checks, and recover from simulated failures.

By the end: Distinguish a proposed action from execution and observation, enforce permissions separately, and choose a bounded recovery strategy.

Start with the exampleJump to the experimentRead the explanation
Prerequisites and path options

Why learn this? Build assistants that inspect tool results and recover from failures instead of treating every returned value as trustworthy.

01 · TRY

Start with something concrete.

You ask a calculator for a product. It might return an answer, an error, or the result of an earlier question. Before using the number, you need to know what actually ran and what its result establishes.

Try the interactive model
02 · PREDICT
Before you experiment

A tool call is valid JSON and has correctly typed arguments. Does that guarantee it answers the user’s question?

A tool call is valid JSON and has correctly typed arguments. Does that guarantee it answers the user’s question?
03 · EXPERIMENT

Change one thing. Watch what follows.

Loading this browser experiment…

04 · UNDERSTAND
READ & CONNECT

Understand what changed.

Back to the experiment

A shorter explanation, with the original worked examples available below. Both explain the same experiment and its limits.

Same learning goal in both modes: Distinguish a proposed action from execution and observation, enforce permissions separately, and choose a bounded recovery strategy.

The idea in everyday terms

A model can suggest a tool call, but another part of the system must decide whether the call is allowed and properly formed. A tool result then needs interpretation: it may be irrelevant, stale, incomplete or associated with a different request.

What to watch in the experiment

Follow the loop from observation to proposed call, runtime check, tool result and next decision. Compare bounded retry and alternative-tool choices after a failure. Check both what happened and whether that outcome answers the original request.

What this does not establish

Tool access does not supply authorization or make every result trustworthy. These local tool contracts are bounded teaching examples, not an unrestricted autonomous service.

Open the full worked examples, calculations and limitations

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

An answer is not an action

A predictor takes an input and produces an output. An agent operates over time: observe → decide → act → observe again. It keeps enough state to connect earlier decisions with later consequences. A model might propose a tool call, but another component must actually execute it.

Calculators, compilers, databases, search systems, and robotic controls perform different operations. The controller selects a tool and supplies arguments. The runtime checks those arguments and its own authorization rules before dispatching the operation. It returns a result or an error as an observation.

In this lab you are the controller. The calculators are local programmed functions, and the failures are simulated. There is no trained tool-selection policy or language model behind the buttons. This separation lets you inspect what a future learned controller would have to handle.

Follow one complete call

The question is 347 × 28. A structured call names the action and carries numerical inputs. The runtime permits calculation, checks the argument types, and executes multiplication. A successful response contains 9,716, a copy of the request, and a receipt revision. The final arithmetic check decomposes the product: 347 × 20 = 6,940 and 347 × 8 = 2,776; their sum is 9,716.

That number becomes evidence in the current history. It is not a weight update. Training would require a separate procedure that constructs a learning objective, computes or estimates an update, and retains changed parameters. Reusing an observation as context and learning from it through gradients are different operations.

Check more than the format

CheckWhat it can establishWhat it does not establish
JSON and argument schemaThe request has the supported fields and input typesThat the task is useful or authorized
Runtime permissionThis action is allowed in this environmentThat it will succeed or answer the question
Response schemaThe result has the expected fields and typesThat it belongs to this request or is true
Receipt and provenanceWhich request and revision produced the evidenceThat an external source is complete or accurate
Task-specific verificationThe particular property being checkedEvery other property or a general reliability guarantee

The wrong-operation preset adds 347 and 28. It is well formed and returns correct arithmetic for the request, yet does not answer the assigned multiplication question. The old-receipt scenario returns arithmetic from another request. Its format is valid, but its provenance does not match. Arithmetic does not expire; a cached observation can simply be attached to the wrong task. In changing domains, timestamps and source freshness matter too.

Even a current response can be incomplete. An empty search result, for example, does not prove that the requested fact does not exist. Interpret observations within what the tool actually measured or searched.

Let the failure determine the next action

Retry can help with a transient failure, such as the first-call timeout here. Repair the arguments when the schema rejects them. Switch to a compatible tool if one repeatedly returns malformed results. Inspect the request, source, or execution history when the evidence is unclear. Stop when the task is complete, permission is missing, or the attempt allowance is exhausted.

This lab caps each trial at three attempts, including invalid calls. It keeps the request, tool, checks, and observation for each attempt. A real controller needs an execution history for the same reason: “the calculation was requested” is a hypothesis about what should happen; “a matching result was received” is an observation about what did happen.

A timeout does not always mean an operation had no effect. Retrying a read-only calculation is different from retrying a payment or a file deletion that may already have completed. Systems with external effects need status checks and mechanisms that prevent duplicate effects before they retry. The denied-action preset performs no deletion: the runtime blocks it independently of the controller’s proposal.

Design a debugging loop

The source exercise asks for an observation–action sequence to debug a failed Python test. Before opening the worked answer, describe the first action, the evidence it returns, and one recovery branch.

Worked debugging sequence and failure branch
  1. Run the relevant test within the permitted environment. Preserve the command, exit status, and output.
  2. If the test runs and reports an incorrect value, inspect the failing assertion and relevant code. Form a specific hypothesis before proposing a change.
  3. If the test cannot start because a dependency is missing, treat that as an environment failure, not evidence that the implementation is wrong. Inspect the declared setup or report the missing capability before retrying.
  4. After an authorized change, rerun the failing test and relevant regression checks. Passing only the original test does not establish that unrelated behavior stayed correct.
  5. Record the result and stop at the agreed scope or resource limit. Keep observations, hypotheses, and changes distinct.

This is a design exercise; the website does not execute Python or modify files.

Tools expand what a system can do. Reliable use still depends on choosing a relevant operation, enforcing boundaries outside the model, and interpreting feedback correctly.

05 · APPLY
Check your understanding

A read-only tool returns an old receipt for a different request. What should the controller do?

A read-only tool returns an old receipt for a different request. What should the controller do?
06 · CONTINUE

Take this idea with you.

A reliable tool loop separates proposal, runtime enforcement, execution, and evidence. Recovery should respond to the observed failure and stop within a defined allowance.

My Learning
Explore related topics and the topic map

How this topic connects

Explore this topic on the interactive map

Use these connections to review a foundation or choose where to go next. You can open any topic without completing the others.

What it builds on

  • Policies and observations

    Understand why trying an uncertain option can improve later decisions, even when it costs something now.

  • Verification

    Build practice loops that reward correct results instead of letting a weak checker reinforce mistakes.

You are hereCall a tool, then check the evidence

Propose structured calls, inspect independent runtime checks, and recover from simulated failures.

What it enables

Show 3 more connections