Run section 9 of the notebook to export a JSON, put it next to this page, then open Settings (the gear) to pick it. From a local file:// path use “Choose a local file”.
Every particle has its own colour. Spotlight one (the prediction, by default); then for each earlier particle that pushes it hardest, an arrow is drawn on the spotlight in that source's colour — direction and length are the velocity that source contributes. A steel arrow is the global field, the white one the total. Dashed links point back to each source; solid/dashed rings show the dominant force's f / g fields. Click any particle to spotlight it.
Turn on Hidden in the player. Each token is run through the model across many generated sentences; with scope all words the cloud shows the current layer and animates as the layers advance; with scope only the current input it overlays every layer at once, so you see all the places those words visit throughout the network. Colour = word type: verbs, nouns, adjectives, the / determiners, other. The separate Embeddings player button draws every vocabulary word transparently at its embedding (layer-0) position.
Each token of the input is a particle dropped on the torus at its learned embedding. Particles flow through the layers under the causal prefix-mean interaction; the readout turns each final position into a next-token guess. This runs the trained model exactly (verified to ~1e-6 against PyTorch).
This page treats a transformer the way the paper On Transformer Dynamics (M. J. Latifi Jebelli, 2026) does: not as a stack of matrix multiplications, but as a system of interacting particles. Every token of your sentence is one particle, and the network's layers play the role of time — as depth increases you are literally watching the particles move.
The space. The particles live on a manifold, a smooth space that curves back on itself. Here it is a flat torus — the square wraps around at all four edges — the simplest compact manifold to draw. A word's meaning is nothing more than where its particle sits, and the trained model supplies a learned starting place for each word.
The interaction. What we usually call attention is, in this picture, a two-body interaction law: one fixed rule that, for any ordered pair of particles, says how the first should push the second. Two properties let such a law carry language. It is nonlocal — a particle can be moved by others anywhere on the space, not only nearby ones — and it is nonreciprocal — the pull of particle i on j need not match the pull of j on i. Everyday forces like gravity or springs are symmetric and reciprocal; language is not, because word order and dependency point in a direction. The arrows you see are exactly these forces acting on the highlighted particle.
Direction and universality. The push is also causal: a particle is influenced only by the tokens before it. That one-way pattern of who-acts-on-whom is what the paper calls an attention digraph, and its main result is that a small, fixed set of interaction parameters — the same few no matter how large the space — is universal: it can reproduce essentially any such digraph. So even this modest particle system is expressive enough to stand in for the attention of a real transformer.
Reading the answer. After the last layer the particles have settled into a new arrangement — a geometric summary of the context. The model reads that final configuration off the manifold and turns the resting place of the last particle into a probability over the next word. The output map simply paints the whole torus with that probability for a word you choose.
The example: “the cat uses the ___”. The guess is read from the final the particle — but, because the interaction is nonlocal and directed, that particle's entire journey is shaped by the ones ahead of it. Change cat → owl and only one starting point moves; yet that single change flows through the directed interaction, bends every later step, and leaves the predicting particle resting in a different region of the torus. Same fixed laws of motion, a different initial condition — and so a different pattern of probabilities for the next word.