Predict what an action will change.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

From predicting a label to predicting a consequence

A classifier predicts a label from an input. A world model predicts how an environment changes after an action. The conditioning matters: the same state can lead to different next states depending on what the agent does.

In this deterministic grid, a state is a position and an action is a direction. Boundaries and walls determine the next position. Our small model learns a table of observed state–action outcomes. Before a pair is observed, it uses a movement prior that ignores walls. The table is learned from data, but the prior and learning rule are programmed; there is no neural encoder in this experiment.

Because the environment is deterministic, repeated observations of the same pair agree. A noisy environment would require a distribution over outcomes rather than treating one observation as certainty. An unobserved pair can be predicted correctly by the prior, so agreement alone is not proof that the model learned that transition.

Retain the state that the task needs

An image-based robot may use a neural encoder to transform observations into a compact state representation. A learned transition model then predicts the next latent state given an action. Predicting useful latent quantities can be cheaper than rendering every pixel of a future image.

Compression can also discard something essential. Omitting a small obstacle may make the predicted route look clear. The source’s sliding-on-ice example asks what position alone misses: velocity or motion direction, whether the robot is still sliding, friction or surface type, and relevant collisions can affect the next outcome. Two observations with the same position may require different predictions.

If the current observation does not expose a needed variable, a history or a state estimate may help. The requirement is not “store everything”; it is “retain enough information for the prediction and task.”

Plans can drift away from reality

A rollout repeatedly feeds a predicted state into the next prediction. An early model error can therefore change all later imagined states. High average one-step agreement does not guarantee a correct long route, especially when a few critical transitions are wrong.

The lab’s open-loop strategy computes a full action list once. If a move is blocked, later commands still come from the old plan. The trace’s imagined states belong to that rollout; actual states come from exact execution. This makes the divergence visible.

Model-predictive control plans ahead, executes the first action, observes the resulting state, and replans. Our trial additionally updates a temporary transition table from each observed move. Both strategies receive those observations, but only the replanning strategy uses them to choose a new route. Neither trial changes the saved model.

Replanning costs extra searches and still has a real-move limit. An inaccurate model, missing state, unreachable goal, or insufficient budget can defeat it. Collecting transition data beforehand can reduce those mistakes, but the same-grid coverage diagnostic is not a held-out generalization test.

Exact rules and learned dynamics have different jobs

When the rules are known, as in chess or a defined puzzle, an exact simulator avoids unnecessary prediction errors. Learned dynamics are useful when exact rules or relevant state are unavailable or expensive to model directly.

The source describes MuZero-style systems that learn representations and dynamics-related quantities useful for search, along with policy and value predictions, without reconstructing every visual detail. The lesson is about task-relevant prediction, not a claim that our transition table implements that architecture. Judge a world model by the decisions it supports and the errors it produces, not only by how plausible its imagined future looks.