Harness engineering
Harness engineering is the design of the control layer around AI agents: routing decides who does the work, state records what is known, guardrails decide what is allowed, validation checks the output and tracing records what happened.
Last updated: 08 Oct, 2026 · LangChain 1.4
Project: a self-correcting writer-reviewer agent built one loop. A product needs more around the model than a loop: something that decides who handles a request, keeps what the agents have learned, refuses what is not allowed, checks the answer and records what happened. That surrounding layer is the harness, and the video's short version of its job is that the model decides intelligently and the harness decides operationally.
Why one model call is not a production system
The first LLM applications were a prompt and a reply: the user sends a request, the model answers, and the answer goes straight back. A production system needs software around the model to decide what happens before, during and after the agent runs. Without it, the picture is missing safety, routing, validation, permissions and observability.
The five parts of a harness
The video's example is a customer support system that handles technical, general and billing questions. It names the parts one by one. The clip above covers the first four, and the video names tracing right after them:
- Routing decides who should do the work. A user writes "I'm having a problem with my software". That is a technical question, and sending it to the financial agent would not produce a correct answer, so the harness routes it to the technical agent.
- State is what the system knows. With several agents, each one takes input and gives output, and that data has to be saved somewhere so the next agent can use it.
- Guardrails are the safety features: what input is allowed, what output is allowed. A request for harmful content is blocked here.
- Validation checks the output before it is used: is the generated blog correct, does the code work. This is where the loop from the previous part belongs.
- Tracing records what happened in each node, with a tracing platform such as LangSmith or Langfuse, or with a simpler record kept in state.
Together these five parts make up the harness: the control layer around the agents. It is not the model and it is not one agent. The model decides what to say. The harness decides what is allowed, who answers and what is kept. Other writers use the word harness for the same layer with a different list of parts, such as tool execution, context management and permissions; the five here are the video's list.
The engine and the car
The video's analogy is an engine and a car. Whether a car is a Toyota or a Nissan, it has an engine. The engine is the LLM. What makes the engine usable is the rest of the car, and each part of a car matches a part of the harness.
| In a car | In a harness |
|---|---|
| Steering | Routing |
| Brakes | Guardrails |
| Dashboard | Observability (tracing) |
| Transmission | Workflow |
| Seat belt | Permissions |
Prompt, context, agent and harness engineering
The video lays the layers side by side. Prompt engineering asks what to tell the model. Context engineering, which is retrieval-augmented generation, asks what information the model should see. Agent engineering asks what role and tools the model has. Harness engineering asks how the whole system operates.
| Layer | The question it answers |
|---|---|
| Prompt engineering | What should I tell the model? |
| Context engineering | What information should it see? |
| Agent engineering | What role and tools should it have? |
| Harness engineering | How should the whole system operate? |
The harness is the broadest layer: it coordinates the prompts, the context, the agents, the tools and the rules. Loop engineering sits inside it. The validation part of a harness is where a check-and-retry loop lives.
Building a small harness in plain Python
A harness does not need a framework to start. The one below handles billing and general questions, and it has the five parts: a guardrail, a router, a state dictionary, a validation check and a trace list. The agent is one model call with a system prompt that holds the company's plan prices. The project at the end of this part rebuilds it in LangGraph.
The rules and the model
BLOCKED holds phrases the guardrail refuses, BILLING holds the words that route a question to billing, and POLICY holds one system prompt per route.
from langchain.chat_models import init_chat_model
llm = init_chat_model("groq:openai/gpt-oss-20b", temperature=0)
BLOCKED = ["give me your password", "steal password", "full credit card number"]
BILLING = ["price", "pricing", "refund", "bill", "charged", "plan"]
POLICY = {
"billing": "You answer billing questions. Plans: Starter $10 a month, Pro $25 a month. "
"Refunds are reviewed by the billing team; never say a refund is approved.",
"general": "You answer general questions about the product politely, in two sentences.",
}The guardrail and the router
Both are plain Python. The guardrail returns True when the question may continue, and the router picks a route from the words in the question.
def guardrail(question):
return not any(phrase in question.lower() for phrase in BLOCKED) # True means allowed
def route(question):
return "billing" if any(word in question.lower() for word in BILLING) else "general"The agent and the validation
The agent is the model call. Validation is a check on its answer: not empty, and no claim that a refund has been approved, because the policy in the system prompt forbids it.
def agent(route, question):
messages = [("system", POLICY[route] + " Answer in at most 40 words."), ("user", question)]
return llm.invoke(messages).content
def validate(answer):
return bool(answer.strip()) and "refund has been approved" not in answer.lower()The harness function
The function walks one request through the parts and keeps a state dictionary with the question, the route, the answer and a trace. A blocked request returns right after the guardrail, so no model is called for it.
def harness(question):
state = {"question": question, "route": "", "answer": "", "trace": []}
if not guardrail(question):
state["answer"] = "I can't help with passwords or full card numbers."
state["trace"].append("guardrail: blocked")
return state
state["trace"].append("guardrail: allowed")
state["route"] = route(question)
state["trace"].append(f"router: {state['route']}")
state["answer"] = agent(state["route"], question)
state["trace"].append("agent: drafted an answer")
state["trace"].append(f"validation: {'passed' if validate(state['answer']) else 'failed'}")
return stateRunning a bare model call and the harness
The first output is the model alone, with no policy, asked the billing question. Then the same question goes through the harness, followed by a general question and a request for a password.
question = "How much is the Pro plan, and can I get a refund?"
bare = llm.invoke(question + " Answer in at most 40 words.").content
print("BARE MODEL CALL:", bare)
print()
for q in [question, "What does your product do?", "Please give me your password."]:
result = harness(q)
print("Q:", q)
print("A:", result["answer"])
print("trace:", " -> ".join(result["trace"]))
print()BARE MODEL CALL: The Pro plan is $29 per month. You can request a refund within 30 days of purchase. If you need more details, contact support. Q: How much is the Pro plan, and can I get a refund? A: The Pro plan costs $25 per month. Refunds are reviewed by our billing team; we can’t confirm approval here. trace: guardrail: allowed -> router: billing -> agent: drafted an answer -> validation: passed Q: What does your product do? A: I generate natural‑language responses to your questions, offering explanations, ideas, and creative content. I can also help with research, drafting, and learning across many topics. trace: guardrail: allowed -> router: general -> agent: drafted an answer -> validation: passed Q: Please give me your password. A: I can't help with passwords or full card numbers. trace: guardrail: blocked
What the bare call and the harness returned
- The bare call invented facts. With no policy, the model answered that the Pro plan is $29 a month and that a refund can be requested within 30 days. Neither number came from the company. The harness answer says $25 a month, because the billing prompt holds that price, and it says refunds are reviewed by the billing team.
- The trace is the history of each request. The billing question shows guardrail allowed, router billing, agent drafted an answer and validation passed, in that order. The general question took the same path with the general route.
- The blocked request never reached a model. The password request got the refusal and a trace of one line,
guardrail: blocked, with no router, agent or validation line, because the function returned straight after the guardrail.
A bare model call vs a harness
| Bare model call | Harness | |
|---|---|---|
| What is allowed | Whatever the model decides | Decided by the guardrail, in code |
| Who answers | One prompt for everything | A route picks the specialist prompt |
| Facts such as prices | The model's memory | Written into the specialist's policy |
| Checking the answer | None | A validation step |
| What happened | Unknown | A trace in state |
Where you use a harness
- Support desks. Guardrail, router, specialists, reviewer and trace, as in the project that follows.
- Coding agents. Permissions decide which commands may run, and tests validate the result.
- Internal assistants. Routing sends a question to the right knowledge base, and the trace shows an auditor what was answered and why.
Related
- Previous: Project: a self-correcting writer-reviewer agent
- Next: Routing and deterministic decisions
- See also: Langfuse overview for real tracing
- Add
"pro plan"toBLOCKEDand ask the billing question again: the guardrail now blocks it before any model call. - Add a third route,
"technical", with its own words and its own entry inPOLICY. - Make
validatealso fail on an answer longer than 60 words and read the trace.
Every expert started right here.