Anatomy of a good loop
A good loop has six parts: a goal, an action, an observation, an evaluation, feedback and a stopping rule, which the video writes as Loop = Goal + Action + Evaluation + Feedback + Stop condition.
Last updated: 08 Oct, 2026 · LangChain 1.4
Verifiers and stop conditions covered two of the parts. This lesson names all six, places them in one picture and then builds a loop where each part is a separate piece of code, so a weak part is easy to find.
The six parts of a loop
The video lists what a good loop needs: the goal, the action, the observation, the evaluation, the feedback and, most importantly, the stopping rule. Without a stopping rule it is not a loop worth building. Its formula puts the parts together:
- Goal. The result the agent must reach, stated so it can be checked.
- Action. What the agent does in each attempt: write code, draft a reply, build a plan.
- Observation. What comes back from the action: a result, an error, a signal.
- Evaluation. How the loop knows whether the result passed. This is the verifier.
- Feedback. What to improve next time, written so the next action can use it.
- Stopping rule. When the loop ends, whether it passed or not.
In the formula the observation sits inside the evaluation, since the evaluation reads the observation. Keeping them apart in code is still useful: the observation is a plain fact, and the evaluation is a judgement about it.
Speed or quality
The video's trade-off is the same one as before, put as a choice. A non-loop agent answers once: fast, low cost, no automatic correction, good for low-risk tasks. A loop agent tries more than once, checks the output and uses the feedback to improve: slower, higher cost, better for multi-step tasks. Where speed matters, such as a chat reply, the non-loop agent fits. Where quality matters, such as the output of a coding agent, waiting for a few more attempts is acceptable.
A coding loop and a travel planner
The video gives two examples. In the coding loop the agent writes the code, runs the tests, reads the errors, fixes the code and runs the tests again. The goal is working code, the observation is the test result, and the stopping rule is that all the tests pass or the agent reaches its maximum number of repair attempts.
The travel planner is a harder case because a model can invent. It may propose a flight that does not exist, so the loop checks that the plan is within the budget, the dates are valid, the travel time is realistic, the hotels are available and the user's constraints are followed. The next section builds a small version of that planner.
Building the planner as six parts
The goal is one day in Lisbon with six items, and the budget is $120. The model's answer is a structured plan: a list of items, each with a cost. Structured output (see Structured output) matters here because the evaluation adds up the costs in code, and code cannot add up a paragraph.
The plan schema
with_structured_output(Plan, method="json_schema") makes the model return a Plan object. UsageMetadataCallbackHandler collects the token usage of every call it is attached to, so the loop can report its cost. A call is attached by passing config={"callbacks": [usage]}, and usage.usage_metadata then holds one token record per model name.
from pydantic import BaseModel
from langchain.chat_models import init_chat_model
from langchain_core.callbacks import UsageMetadataCallbackHandler
class Item(BaseModel):
name: str
cost_usd: float
class Plan(BaseModel):
items: list[Item]
llm = init_chat_model("groq:openai/gpt-oss-20b", temperature=0)
planner = llm.with_structured_output(Plan, method="json_schema")
usage = UsageMetadataCallbackHandler()The goal and the action
The goal is a plain string. The action calls the planner and, after a failed attempt, adds the feedback to the prompt.
GOAL = ("Plan one day in Lisbon for a visitor: a hotel night, breakfast, lunch, dinner, "
"one museum ticket and a metro day pass. Give each item a cost in US dollars.")
BUDGET = 120
def act(goal, feedback):
prompt = goal + (" Fix this from your last plan: " + feedback if feedback else "")
return planner.invoke(prompt, config={"callbacks": [usage]})The evaluation
The evaluation returns an observation (how many items, what total) and a list of problems. It checks that every required item is present and adds up the costs itself.
def evaluate(plan):
total = sum(item.cost_usd for item in plan.items)
names = " ".join(item.name.lower() for item in plan.items)
problems = [f"add the {w}" for w in ["hotel", "breakfast", "lunch", "dinner", "museum", "metro"] if w not in names]
if total > BUDGET:
problems.append(f"total is ${total:.0f}, over the ${BUDGET} budget by ${total - BUDGET:.0f}")
return f"{len(plan.items)} items, total ${total:.0f}", problemsThe loop with each part labelled
run_loop is written around the six parts, so each comment in the code names one of them. The stopping rule is two-fold: the goal is met, or the attempts are used up.
def tokens_used():
return sum(m["total_tokens"] for m in usage.usage_metadata.values())
def run_loop(goal, act, evaluate, max_attempts=3):
feedback, spent = "", []
for attempt in range(1, max_attempts + 1):
result = act(goal, feedback) # Action
observation, problems = evaluate(result) # Observation and Evaluation
spent.append(tokens_used())
print(f"attempt {attempt}: {observation}; problems: {problems or 'none'}")
if not problems: # Stopping rule: goal met
return result, True, spent
feedback = "; ".join(problems) # Feedback for the next action
return result, False, spent # Stopping rule: attempts used up
plan, passed, spent = run_loop(GOAL, act, evaluate)
print("passed:", passed)
for item in plan.items:
print(f" {item.name}: ${item.cost_usd:.0f}")
print("tokens after each attempt:", spent)attempt 1: 6 items, total $222; problems: ['total is $222, over the $120 budget by $102'] attempt 2: 6 items, total $120; problems: none passed: True Hotel night: $58 Breakfast: $10 Lunch: $15 Dinner: $20 Museum ticket: $12 Metro day pass: $5 tokens after each attempt: [426, 1099]
What the six parts did
- Attempt 1. The action returned six items totalling $222. The evaluation added the costs in code and sent back "total is $222, over the $120 budget by $102". Every required item was present, so the budget was the only problem.
- Attempt 2. The model brought the total down to $120, which is within the budget, so the evaluation returned no problems and the stopping rule ended the loop with
passed: True. The plan lists a $58 hotel night, meals of $10, $15 and $20, a $12 museum ticket and a $5 metro pass. - The cost is counted. The token counts are cumulative: 426 after the first attempt and 1099 after the second, so the retry cost 673 tokens on top of the first attempt's 426.
Non-loop agent vs loop agent
| Non-loop agent | Loop agent | |
|---|---|---|
| Answers | Once, then stops | Works, checks, improves and repeats |
| Strength | Fast and simple, lower cost | Reliability on multi-step tasks |
| Cost | One call | One call per attempt, as the token counts above show |
| Best for | Basic answers, drafting ideas, low-risk content | Plans with constraints, code, anything checkable |
Controls every loop needs
The slides close the topic with the minimum controls for a loop in production, and the rule that goes with them: never design an open-ended loop, because every loop needs a stop condition and a fallback path.
- A maximum number of attempts.
- A token or cost budget.
- A quality evaluator.
- Safety or policy checks, which the Guardrails lesson builds.
- Human approval for risky actions.
- Logs, so a failed run can be read afterwards.
Where you use these parts
- Agent frameworks. A coding agent's test run is the evaluation and its repair limit is the stopping rule.
- Planners. Budget, dates and availability are checks run in code on the model's structured plan.
- Review pipelines. A reviewer's verdict is the evaluation and its notes are the feedback.
Related
- Previous: Verifiers and stop conditions
- Next: Project: a self-correcting writer-reviewer agent
- Reference: LangChain chat models
- Lower
BUDGETto90and read how many attempts the loop needs. - Add
"taxi"to the list of required items inevaluateand add a taxi to the goal. - Change
max_attempts=3to1and see what the loop returns when it cannot retry.
Slow is fine. Stopping is the only problem.