ELOC CYBER
← Lessons 4–9

LESSON 5 / HARNESSES · 50–60 minutes

How does an LLM harness work?

A harness manages the conversation with an LLM and runs its tools. The assignment is to build a harness in VCC and run it in the terminal.

The examples run in your browser. The coding assignments run in your VCC workspace and use the classroom AI allowance.

  • Trace the model → tool → result → model loop.
  • Identify what the model requests and what the harness executes.
  • Build and test a small terminal harness in VCC.

01 / TWO PIECES WORKING TOGETHER

What is a harness versus an LLM?

The LLM predicts tokens. The harness is the software around it: it prepares messages, offers tools, checks requests, runs allowed actions, returns results, and decides when the process stops.

Together, a model and a harness that lets it choose actions form the kind of agent we use in VCC. A chat app can have a simple wrapper. An agent harness adds a loop where the model can request actions, observe what happened, and choose the next step.

The harness runs the tools and returns their results to the LLM

This example follows one calculator task. The LLM requests the operation; the harness checks and executes it. A task can repeat the tool cycle several times before the model supplies a final answer.

Sequence of messages between a user, a harness, an LLM and a calculatorA task goes from the user to the harness. The harness sends messages and tool descriptions to the LLM. The LLM requests a calculator operation. The harness validates and executes it, receives 391, and sends that result back to the LLM. The final answer returns through the harness to the user.YouHarnessLLMTool1 · Task2 · Messages + tool descriptions3 · Requested tool + arguments4 · Validate name and arguments5 · Execute allowed function6 · Actual tool result: 3917 · Tool result + updated context8 · Final answer9 · Return answer and stop

Swipe or scroll sideways to view the whole diagram.

A harness may include a router. It could use simple rules, a trained classifier, or another model call to choose a tool, workflow, or specialist. A separate classifier is one possible design; a harness does not have to contain one.

QUESTION If an LLM writes “I read your file,” which evidence would show the file was actually read?

02 / FOLLOW ONE REQUEST

How tool calls work

FOLLOW THE HARNESS LOOP

    The model requests are scripted for this browser demo. The router uses visible rules, and the calculator really executes JavaScript. The downloaded harness below makes actual LLM requests in your VCC terminal.

    1. Step through the arithmetic task. Point out the model’s requested operation, the harness’s argument check, and the actual tool result.
    2. Try the notes task. The harness retrieves a fixed class note, then provides it as data for the next model turn.
    3. Try the unknown tool and repeated-step cases. Explain why the harness stops.

    03 / ASSIGNMENT

    Build your own harness around an LLM.

    Use the VibeCodeCentral app to build a small harness, then run it with the classroom LLM in your terminal. Start with a calculator because its result is easy to check.

    1. Open VCC Projects ↗ and create My first harness. Stay in VCC sandbox.
    2. In Build & activity, give Ask agent the build prompt below. Wait for its checks.
    3. Open coding workspace → Start workspace → Bash. Use pwd to confirm the project folder. Bash runs commands; the OpenCode tab takes instructions to the coding agent.
    4. Run the offline fixture first, then one real model run. Inspect the trace before changing anything.
    5. Change the system prompt or add one small read-only tool. Predict the effect, rerun, and compare.
    Build my Lesson 5 harness in a new lesson5 folder without changing the starter app. Use Node.js 20+ and no added packages. Download the two class starter files from /lab/agents/downloads/harness.mjs and /lab/agents/downloads/tools.mjs into that folder and inspect them. Preserve --offline as a clearly labeled fixed 17 × 23 fixture. The live mode must use the existing VCC gateway at http://models.vcc/v1/chat/completions and the classroom model from /etc/opencode.json, without personal API keys. Keep system/user/assistant/tool roles and tool_call_id links. Expose only calculate, validate names and numeric arguments, return tool errors, stop after four model requests, and set an output limit. Log the requested tool, actual result, final answer, and provider usage if available. Add meaningful tests for arithmetic, division by zero, invalid inputs, and the stop limit using a fake model endpoint. Run those tests and the existing project tests/build. Tell me the Bash commands and files changed.
    Get the starter directly

    You can inspect and copy the files, or use these commands in your project’s Bash terminal. Use a fresh project/folder so existing work is preserved.

    mkdir lesson5
cd lesson5
curl -fSLO /lab/agents/downloads/harness.mjs
curl -fSLO /lab/agents/downloads/tools.mjs
node harness.mjs --offline
node harness.mjs "Use the calculator to multiply 17 by 23, then explain the result."

    Download harness.mjs · Download tools.mjs. Keep the two files in the same folder. The live command works inside a running VCC workspace; a laptop terminal does not have the internal classroom gateway.

    04 / CHECK YOUR HARNESS

    What the harness controls

    The harness checks a tool request before executing it

    Tool schemas describe the available functions. The harness still validates the actual name and arguments, and applies its own request and output limits.

    Allowed tool and argument validationThe harness checks the model tool request. A known tool with valid arguments can execute. An unknown tool or invalid arguments returns an error without running the function.Model tool requestName + argumentsAllowed tool?Valid arguments?YesRun the functionReturn actual outputNoReturn an errorNo function execution

    Swipe or scroll sideways to view the whole diagram.

    In a successful live run, find these pieces in the terminal:

    1. Harness: packages system and user messages plus a tool schema.
    2. Model: returns a structured request for calculate with operation and numbers.
    3. Harness: validates the name and arguments, executes the JavaScript function, and attaches the result to the matching tool call ID.
    4. Model: reads that result and writes a final answer.
    5. Harness: stops on final text or its request limit.

    Try division by zero and an unsupported tool name in your tests. Check that neither runs an arbitrary command. For the loop test, use a fake model that always requests another tool; it should stop at the configured cap.

    If you add a class-notes tool, keep its data in a fixed text string first. The tool result should identify where that note came from. The harness should decide which functions are allowed; generated text should never be evaluated as arbitrary JavaScript.

    05 / SUBMISSION

    Review and submission

    Who executes the calculator function?

    Does every harness need a separate classifier model?

    Turn in: your harness files or Git commit, a live trace showing 17 × 23 = 391 from the tool, one error-handling test, one stop-limit test, and a short explanation of your change.

    Finish in VCC: inspect git diff, run the relevant tests/build, commit a version you can explain, then select Save checkpoint and Save and stop.

    Review: If we kept the same LLM but removed its harness, which abilities would disappear? Name one place where your harness, rather than the LLM, makes a decision.

    Instructor notes & sources

    Allow about 5–10 minutes for the opening model, 15 for the browser investigation, 20–25 for the VCC activity, and 5 for the review questions. Students can work in pairs and compare their answers. Provision VCC access before class. The demos need no sign-in, API key, or paid inference. If class AI is unavailable, students can inspect demos, run the offline harness, and write expected traces; the live assignment still needs a later model run.

    Assess the prediction, trace, and explanation along with the result. The small models, invented player data, fixed chat replies, and scripted browser requests are labeled at the point of use.

    Further reading: Attention Is All You Need · Chat templates · Agent workflows and routing · Switch Transformers.