LLMOpsThe LLMOps lifecycle · MLflow 3 · FastAPI · Docker · AWS/Azure
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
50 small wins to finish your pathNext lesson →

Node two: the auditor

The second node is the audit itself. It takes the transcript and on-screen text, retrieves the rules that bear on them from Azure AI Search, asks GPT-4o to compare the ad against those rules, and writes back a list of violations in a fixed JSON shape.

Last updated: 15 Oct, 2026 · LLMOps

Node one turned the video into text. This node is where the compliance decision is made, and it is a RAG step: retrieve the relevant rules, then generate a judgement grounded in them.

The auditor: retrieval and the LLM · from the End to End Multimodal LLMOps Project video · 2:14:00 to 2:17:00

Retrieve, then judge

python
def audit_content_node(state: VideoAuditState) -> dict:
    log.info("auditor node: querying the knowledge base and the LLM")

    # the ad's own words become the query against the rules index
    ad_content = state["transcript"] + "\n" + state["ocr_text"]
    rules = store.similarity_search(ad_content, k=3)   # the FTC passages that apply

    # the model is asked for the fixed violation shape, not free text
    answer = llm.invoke(audit_prompt.format(
        rules="\n".join(r.page_content for r in rules),
        transcript=state["transcript"],
        ocr_text=state["ocr_text"],
    ))
    issues = parse_violations(answer)   # -> list of {category, severity, description}
    status = "fail" if issues else "pass"
    return {"compliance_results": issues, "status": status,
            "final_report": format_report(issues)}

The model is GPT-4o on Azure, given the retrieved rules and the ad's content, and asked to return violations in the ComplianceIssue shape from the state. Asking for that fixed JSON is the enforcement the TypedDict did not do: the output is parsed into the list the rest of the system expects, and an ad with no violations yields an empty list and a pass.

The auditor node as a retrieve-then-judge flow: the transcript plus OCR text becomes a query against Azure AI Search, which returns the rules that apply; those rules and the ad go to GPT-4o, which compares the ad against them and returns violations as JSON. A strip below says each violation carries a category, a severity and a description, that no violations means an empty list and a pass, and that the fixed shape is the contract.

Why this is the lifecycle in one node

  • Retrieval decides what the model sees. If the right rule is not retrieved, the violation is missed, which is the context-recall problem from What to evaluate: RAG vs a plain LLM.
  • Faithfulness matters. The verdict has to follow the retrieved rules, not the model's own opinion of the ad.
  • Structured output is the contract. A category, a severity and a description per issue is what makes the result something the API can return and a dashboard can count.
Watch out. A clean audit with an empty violations list is not proof the ad is compliant; it may mean retrieval missed the rule that applies. The evaluation discipline from the tracking part, a golden set of ads with known verdicts, is what tells you the auditor is right, not merely quiet.
Try it yourself
  • Explain how a missed retrieval and a truly compliant ad produce the same empty result, and how you would tell them apart.
  • Name the three fields every violation must carry and why free text would not do.

Little by little, you're building something great.