01 / NEXT-TOKEN PREDICTION
A sentence with several possible endings
Write an ending for “The dragon opened its lunchbox and found a…” A sandwich? A tiny knight? Yesterday’s homework? Compare with a partner. There are many possible continuations, and the words before the gap help you choose.
A language model gives possible next tokens scores based on its training and the context it has received. A token is a piece of text: a word, part of a word, or punctuation. After one token is chosen, it joins the context. The model makes another prediction using that longer input.
One prediction becomes part of the next input
The next round uses “The dragon found a sandwich” as context.
Try adding “For lunch” to the beginning of your sentence. Did that change your ending? Explain which information you used.
02 / THE NETWORK
Follow one prediction
The model first represents tokens as lists of numbers. Attention combines information from the available token positions. Feed-forward layers then transform those numbers, and an output layer scores the next-token candidates. Larger models repeat attention and feed-forward steps in many layers.
- Represent tokens
- Combine context
- Calculate activations
- Score candidates
- Choose the cat sentence. Before selecting Next stage, predict which ending will receive the largest score.
- Follow all four stages. At the neural-network stage, compare the four neuron values.
- Select Append the selected token. Why does the next prediction change?
- Reset with the basketball or robot context. Compare the neuron values and candidate bars.
03 / WEIGHTS AND ACTIVATIONS
Why do the neurons light up?
A weight is a number learned during training that changes how an input contributes to the next calculation. An activation is a value produced while the network processes a particular input. In the diagrams, a brighter neuron represents a larger activation.
Think of a sound mixer. Its saved settings help determine what comes out, but the sound levels change with the music. Similarly, this model keeps its weights while the input and activations change. The analogy describes changing numbers; a language model is not literally hearing music.
A neuron is a small calculation within a much larger network. We cannot label one bright dot “the sandwich thought” just by seeing it light up. Useful patterns can involve many neurons, and one neuron can participate in different patterns.
Pair activity: One student explains what changes when we switch from the cat sentence to the robot sentence. The other explains what changes during training. Use the words input, weight, and activation.
04 / A RUNNING MODEL
Watch a pretrained model
The separate visual aid runs a small model trained to continue stories. Its display follows an actual calculation and lets you examine a multiplication using the model’s own weights. You can pause the visual explanation without needing to understand every equation.
- Select Load pretrained model, then Process prompt / reset. The opening is “Once upon a time, there was a little”.
- Find the selected next token. Select Generate next token three times and read the growing story.
- In the calculation panel, select Play calculation. The slower display replays arithmetic from the completed prediction. The live status identifies when the model is actually computing.
- Try “One day, Lily found a” as the opening. Record the first selected token for each opening and one change you see in the activity.
This is a story model, so its writing can be strange or repetitive. It runs on your browser’s CPU. The viewer shows that model’s own values; it does not show the internals of the classroom chatbot.
Optional VCC activity
In a VCC sandbox project, ask the coding agent to make a simple next-token game using fixed candidate lists. It should label its probabilities as invented. Compare that scripted game with the pretrained model’s calculated predictions.
In a new lesson7 folder, build a small next-token guessing game with three sentence openings and four fixed candidate endings for each. Label the candidate probabilities as invented classroom examples, not a trained model. Let me choose a candidate, append it, and reset. Use no API or extra packages. Keep the existing app working. Run its checks and tell me how to open the game.
Check all three openings, the append button, and reset. Save your project and stop the workspace when you finish.
05 / REVIEW
Explain one prediction
Turn in: the two story openings you tried, their first selected tokens, and a sketch showing context → network → token → longer context. Add two sentences explaining weights and activations. If your predicted ending differed from the model’s, explain one possible reason.
Next lesson: Can a model have many clusters of neurons while using only a few clusters for one token?
Instructor notes & sources
Suggested pacing: 5 minutes for the lunchbox sentences, 12 for the small network, 8 for weights and activations, 15 for the separate viewer, and 5–10 for review. Pair middle-school students for the viewer; older students can inspect the numerical terms. No GPU setup, profiler, or memory-bandwidth calculation is required.
Open the viewer before class to cache its model files. If downloads are unavailable, complete the small local model and sketch activity, then return to the pretrained viewer later. Assess whether students distinguish calculated predictions, fixed teaching examples, and factual correctness.
Technical references: Attention Is All You Need and TinyStories model. The separate viewer retains the pinned model and independent numerical checks.