LAB 02 / REINFORCEMENT LEARNING LIVE IN YOUR BROWSER

Pac-Man-style maze

Try a move. Get a reward. Learn what to try next.

Help a maze explorer collect every pellet while a ghost gives chase. The network starts with random weights and learns from the game’s rewards. No account or installation needed.

It’s playing now. Watch an attempt, or load a practiced brain to compare.

The game

First move

Last reward 0.00New attempt

1Attempt

0Pellets collected

0.0Attempt reward

0Mazes cleared

Inside the neural network

ACTUAL FORWARD PASS

Positive valueNegative valueBrighter = larger activation

The ring marks the chosen move. A gold ring means a random exploratory move. Output scores estimate future rewards; they are not probabilities. Hover or tap a neuron to inspect its value.

Change one setting

Game speed 1×Speed up practice, then slow down to watch.Random moves 60%More exploration means trying more surprises.Learning speed 0.001Size of the network’s weight adjustments.Care about later rewards 0.95A higher value gives future rewards more influence.

Gradually reduce random moves as it practicesAgent mode

Exploration now 60%Weight updates 0Memories 0

Fresh learning can take many attempts. A practiced brain was trained with this same learning algorithm. Loading it gives you a useful comparison; it is not a scripted player.

Is practice helping?

—Average reward · last 10 attempts

0Completed attempts in this run

Light line: each attempt. Dark line: the recent average. Rewards can improve, dip, or stay flat. We measure results rather than promising that every attempt will win.

What can the agent see?

It reads the measurements below, not the game’s screenshot. Those are the input neurons. Nearby walls, food distances, ghost position, power status, remaining pellets, position, and the previous action guide each turn.

Recent moves and rewards

WHAT MAKES THIS REINFORCEMENT LEARNING?

The game gives feedback, not the right answer.

Observe

The agent reads its surroundings: where it is, what is nearby, and what could help or hurt.

Choose an action

The network scores each possible move. Sometimes the agent explores with a random move instead.

Get a reward

Pellets give +1, clearing the maze gives +6, and being caught costs 3. Power pellets give a bonus and make ghosts edible. Moving closer to food helps a little; each move has a small cost, and hitting walls costs extra.

Adjust the weights

It saves what happened and learns from past moves. Better reward estimates can lead to better choices.

Try this: Load a practiced brain and follow the ghost. Why does the agent sometimes move toward it? Then start fresh and compare low and high exploration over several attempts. Then compare learning mode and watch-only mode. Which changes the network’s weights?

A little more detail: how the learning works

This is a small deep Q-network: input measurements → 24 hidden neurons → 16 hidden neurons → one reward estimate per move. Hidden neurons use tanh; their signed activations are the values shown live.

Training samples past moves from a memory buffer. The target is the reward received, plus a discounted estimate of later rewards when the attempt continues. A second, slowly updated network helps stabilize those targets. This implementation uses Double DQN action selection, Huber loss, and Adam weight updates.

At higher speeds, several moves happen between screen frames. The game and network display the latest decision. Connection brightness reflects the signal through that connection. Hidden neurons have no hand-assigned meaning such as “fear” or “jump.”

This is an original miniature maze game inspired by Pac-Man, with its own layout and a single ghost. It is a small environment for exploring reinforcement learning. Learning and rendering happen on your device. Downloaded brains contain weights and settings, not an account or a game recording.

Further reading: PyTorch’s deep Q-learning tutorial.