Generative AI and Agentic AILangChain 1.4 · LangGraph 1.2 · Python 3.12 or 3.13
Dashboard
0%

Anatomy of a good loop

A good loop has six parts: a goal, an action, an observation, an evaluation, feedback and a stopping rule, which the video writes as Loop = Goal + Action + Evaluation + Feedback + Stop condition.

Last updated: 08 Oct, 2026 · LangChain 1.4

Verifiers and stop conditions covered two of the parts. This lesson names all six, places them in one picture and then builds a loop where each part is a separate piece of code, so a weak part is easy to find.

Anatomy of a good loop and the formula · from the Loop and Harness Engineering Crash Course video · 48:34 to 50:56

The six parts of a loop

The video lists what a good loop needs: the goal, the action, the observation, the evaluation, the feedback and, most importantly, the stopping rule. Without a stopping rule it is not a loop worth building. Its formula puts the parts together:

The simple formula for a good loop
The parts of a good loop. A goal sets the target; an action produces a result; the observation is what comes back; an evaluation judges it; feedback says what to improve and goes into the next action; a stopping rule ends the loop.
  • Goal. The result the agent must reach, stated so it can be checked.
  • Action. What the agent does in each attempt: write code, draft a reply, build a plan.
  • Observation. What comes back from the action: a result, an error, a signal.
  • Evaluation. How the loop knows whether the result passed. This is the verifier.
  • Feedback. What to improve next time, written so the next action can use it.
  • Stopping rule. When the loop ends, whether it passed or not.

In the formula the observation sits inside the evaluation, since the evaluation reads the observation. Keeping them apart in code is still useful: the observation is a plain fact, and the evaluation is a judgement about it.

Speed or quality

The video's trade-off is the same one as before, put as a choice. A non-loop agent answers once: fast, low cost, no automatic correction, good for low-risk tasks. A loop agent tries more than once, checks the output and uses the feedback to improve: slower, higher cost, better for multi-step tasks. Where speed matters, such as a chat reply, the non-loop agent fits. Where quality matters, such as the output of a coding agent, waiting for a few more attempts is acceptable.

A coding loop and a travel planner · from the Loop and Harness Engineering Crash Course video · 51:49 to 53:41

A coding loop and a travel planner

The video gives two examples. In the coding loop the agent writes the code, runs the tests, reads the errors, fixes the code and runs the tests again. The goal is working code, the observation is the test result, and the stopping rule is that all the tests pass or the agent reaches its maximum number of repair attempts.

The travel planner is a harder case because a model can invent. It may propose a flight that does not exist, so the loop checks that the plan is within the budget, the dates are valid, the travel time is realistic, the hotels are available and the user's constraints are followed. The next section builds a small version of that planner.

Building the planner as six parts

The goal is one day in Lisbon with six items, and the budget is $120. The model's answer is a structured plan: a list of items, each with a cost. Structured output (see Structured output) matters here because the evaluation adds up the costs in code, and code cannot add up a paragraph.

The plan schema

with_structured_output(Plan, method="json_schema") makes the model return a Plan object. UsageMetadataCallbackHandler collects the token usage of every call it is attached to, so the loop can report its cost. A call is attached by passing config={"callbacks": [usage]}, and usage.usage_metadata then holds one token record per model name.

python
from pydantic import BaseModel
from langchain.chat_models import init_chat_model
from langchain_core.callbacks import UsageMetadataCallbackHandler

class Item(BaseModel):
    name: str
    cost_usd: float

class Plan(BaseModel):
    items: list[Item]

llm = init_chat_model("groq:openai/gpt-oss-20b", temperature=0)
planner = llm.with_structured_output(Plan, method="json_schema")
usage = UsageMetadataCallbackHandler()

The goal and the action

The goal is a plain string. The action calls the planner and, after a failed attempt, adds the feedback to the prompt.

python
GOAL = ("Plan one day in Lisbon for a visitor: a hotel night, breakfast, lunch, dinner, "
        "one museum ticket and a metro day pass. Give each item a cost in US dollars.")
BUDGET = 120

def act(goal, feedback):
    prompt = goal + (" Fix this from your last plan: " + feedback if feedback else "")
    return planner.invoke(prompt, config={"callbacks": [usage]})

The evaluation

The evaluation returns an observation (how many items, what total) and a list of problems. It checks that every required item is present and adds up the costs itself.

python
def evaluate(plan):
    total = sum(item.cost_usd for item in plan.items)
    names = " ".join(item.name.lower() for item in plan.items)
    problems = [f"add the {w}" for w in ["hotel", "breakfast", "lunch", "dinner", "museum", "metro"] if w not in names]
    if total > BUDGET:
        problems.append(f"total is ${total:.0f}, over the ${BUDGET} budget by ${total - BUDGET:.0f}")
    return f"{len(plan.items)} items, total ${total:.0f}", problems

The loop with each part labelled

run_loop is written around the six parts, so each comment in the code names one of them. The stopping rule is two-fold: the goal is met, or the attempts are used up.

ExampleAPI keyLive run on Groq, openai/gpt-oss-20b
def tokens_used():
    return sum(m["total_tokens"] for m in usage.usage_metadata.values())

def run_loop(goal, act, evaluate, max_attempts=3):
    feedback, spent = "", []
    for attempt in range(1, max_attempts + 1):
        result = act(goal, feedback)                       # Action
        observation, problems = evaluate(result)           # Observation and Evaluation
        spent.append(tokens_used())
        print(f"attempt {attempt}: {observation}; problems: {problems or 'none'}")
        if not problems:                                   # Stopping rule: goal met
            return result, True, spent
        feedback = "; ".join(problems)                     # Feedback for the next action
    return result, False, spent                            # Stopping rule: attempts used up

plan, passed, spent = run_loop(GOAL, act, evaluate)
print("passed:", passed)
for item in plan.items:
    print(f"  {item.name}: ${item.cost_usd:.0f}")
print("tokens after each attempt:", spent)

What the six parts did

  • Attempt 1. The action returned six items totalling $222. The evaluation added the costs in code and sent back "total is $222, over the $120 budget by $102". Every required item was present, so the budget was the only problem.
  • Attempt 2. The model brought the total down to $120, which is within the budget, so the evaluation returned no problems and the stopping rule ended the loop with passed: True. The plan lists a $58 hotel night, meals of $10, $15 and $20, a $12 museum ticket and a $5 metro pass.
  • The cost is counted. The token counts are cumulative: 426 after the first attempt and 1099 after the second, so the retry cost 673 tokens on top of the first attempt's 426.

Non-loop agent vs loop agent

Non-loop agentLoop agent
AnswersOnce, then stopsWorks, checks, improves and repeats
StrengthFast and simple, lower costReliability on multi-step tasks
CostOne callOne call per attempt, as the token counts above show
Best forBasic answers, drafting ideas, low-risk contentPlans with constraints, code, anything checkable

Controls every loop needs

The slides close the topic with the minimum controls for a loop in production, and the rule that goes with them: never design an open-ended loop, because every loop needs a stop condition and a fallback path.

  • A maximum number of attempts.
  • A token or cost budget.
  • A quality evaluator.
  • Safety or policy checks, which the Guardrails lesson builds.
  • Human approval for risky actions.
  • Logs, so a failed run can be read afterwards.

Where you use these parts

  • Agent frameworks. A coding agent's test run is the evaluation and its repair limit is the stopping rule.
  • Planners. Budget, dates and availability are checks run in code on the model's structured plan.
  • Review pipelines. A reviewer's verdict is the evaluation and its notes are the feedback.
Watch out. A loop is only as good as its evaluation. If the check does not look at the budget, the loop will pass a plan that is over it, and the extra attempts will have cost tokens for nothing.
Try it yourself
  • Lower BUDGET to 90 and read how many attempts the loop needs.
  • Add "taxi" to the list of required items in evaluate and add a taxi to the goal.
  • Change max_attempts=3 to 1 and see what the loop returns when it cannot retry.

Slow is fine. Stopping is the only problem.