Routing and deterministic decisions
Routing is the harness step that decides which specialist handles a request, and it should use plain code when the rule is clear and a model only when the request is ambiguous.
Last updated: 08 Oct, 2026 · LangChain 1.4
Harness engineering named routing as the first decision a harness makes. This lesson looks at why a harness routes to specialists at all and at how the routing decision itself should be made, because the cheapest and most testable way is often not a model.
Why route to specialists
One giant agent that has to handle billing, technical problems, sales and refunds is harder to control, test, debug, secure and improve independently. The video's alternative is specialists: a technical agent for technical queries, a billing agent for billing queries and a general agent for general questions. Each gets one job and one prompt.
The video's analogy is a company. A company does not hire one person who knows everything. It has an HR person, a CEO, managers and engineers, each specialised, and they work as a team on a big problem. Specialist agents work the same way, and the router is the step that hands each request to the right one.
Code or a model for each decision
The video's rule for the router is that a model should not make every decision. When the rule is clear, plain code is enough: if the question contains the word "refund", the route is billing. When the question is ambiguous, such as "classify this request into technical, billing or general", plain code cannot do it and a model is the right tool. Using code where code works lowers the cost of the whole multi-agent system.
The router in the video's support demo
The demo's router checks the question against two lists of words. Words such as error, bug, login, API and install mean a technical question. Words such as price, payment, refund and bill mean a billing question. A question with none of them goes to the general agent. The video says that this router is for the demo, and that a real system would use a model to understand the user's intent. The next sections build both, and then a router that uses the two together.
Writing three routers
The keyword router
Two lists of words and one function. The order of the checks decides which group wins when a question matches both, so the technical check comes first here.
TECHNICAL = ["error", "bug", "login", "api", "install", "code", "server"]
BILLING = ["price", "pricing", "payment", "refund", "bill", "plan", "charged", "cost"]
def keyword_route(question):
q = question.lower()
if any(w in q for w in TECHNICAL):
return "technical"
if any(w in q for w in BILLING):
return "billing"
return "general"The model router
The model must return one of three labels, so the schema uses Literal. with_structured_output makes the model fill that schema, which means the router cannot answer with a sentence. calls counts how often a model is used.
from typing import Literal
from pydantic import BaseModel
from langchain.chat_models import init_chat_model
class Route(BaseModel):
route: Literal["technical", "billing", "general"]
router = init_chat_model("groq:openai/gpt-oss-20b", temperature=0).with_structured_output(Route, method="json_schema")
calls = {"model": 0}def llm_route(question):
calls["model"] += 1
prompt = ("Route this customer message to technical (errors, logins, installs, bugs), "
"billing (prices, plans, refunds, charges) or general.\n\nMessage: " + question)
return router.invoke(prompt).routeThe hybrid router
The hybrid router asks the keyword lists first. When a question matches exactly one group, the rule decides. When it matches no group or more than one, the question is ambiguous for the rules, and a model decides.
def hybrid_route(question):
q = question.lower()
groups = [name for name, words in [("technical", TECHNICAL), ("billing", BILLING)] if any(w in q for w in words)]
return groups[0] if len(groups) == 1 else llm_route(question)Comparing the three routers on five questions
Five questions, each with the route a person would pick. Two are built to trouble the keyword lists: one says the app stopped working without using any technical word, and one asks about the cost of the API, which contains a technical word and a billing word.
QUESTIONS = [
"I am getting an API login error.",
"How much is the Pro plan, and can I get a refund?",
"What can your support team help me with?",
"Why did the app stop working after the last update?",
"What does the API cost per month?",
]
EXPECTED = ["technical", "billing", "general", "technical", "billing"]
for name, route in [("keywords", keyword_route), ("model", llm_route), ("hybrid", hybrid_route)]:
calls["model"] = 0
got = [route(q) for q in QUESTIONS]
right = sum(g == e for g, e in zip(got, EXPECTED))
print(f"{name:9} {right}/5 right, {calls['model']} model calls {got}")keywords 3/5 right, 0 model calls ['technical', 'billing', 'general', 'general', 'technical'] model 5/5 right, 5 model calls ['technical', 'billing', 'general', 'technical', 'billing'] hybrid 5/5 right, 3 model calls ['technical', 'billing', 'general', 'technical', 'billing']
What the three routers did
- Keywords: 3 of 5 right, no model calls. The rules got the clear questions right. They sent "Why did the app stop working after the last update?" to general, because it has no technical word, and "What does the API cost per month?" to technical, because the technical check runs first and the question contains "API".
- Model: 5 of 5 right, five model calls. The model read the meaning and routed all five questions as a person would, at the price of one call per question.
- Hybrid: 5 of 5 right, three model calls. The rules decided the first two questions. The other three matched no group or two groups, so the model decided them: the general question, the app update question and the API cost question. The hybrid reached the model's accuracy with three calls instead of five.
Keywords vs a model vs a hybrid router
| Keyword rules | Model router | Hybrid | |
|---|---|---|---|
| Model calls | None | One per request | Only for ambiguous requests |
| Speed | Instant | A model round trip | Instant for clear requests |
| Ambiguous wording | Misroutes it | Reads the meaning | Hands it to the model |
| Testing | Exact and repeatable | Needs example questions | Rules exact, model checked |
| Best for | Clear, stable categories | Free-form requests | Mixed traffic |
Where you use routing
- Support desks. Technical, billing and general questions go to different specialists.
- Assistants with several knowledge bases. A question about contracts goes to the legal index.
- Model selection. A simple question goes to a small model and a hard one to a large model.
Related
- Previous: Harness engineering
- Next: Guardrails
- See also: Multi-agent systems
- Go deeper: Conditional edges
- Add
"crash"and"update"toTECHNICALand see which of the five questions the keyword router now gets right. - Put the billing check before the technical check in
keyword_routeand read what happens to the API cost question. - Add a sixth question of your own to
QUESTIONSand its route toEXPECTED.
Little by little, you're building something great.