Jev: TypeSafe's model that answers questions

The fifth row of that table is middleware, and langchain-typesafe fills it with Jev, a model from TypeSafe AI that answers questions instead of writing sentences.

Your desk asks a model for sentences, but several of its decisions are not sentences: is this urgent, which team should take it, is this refund safe to run. Each of those is a full model call today, with your code reading prose to find a yes. Jev is a model built for that half of the problem: you give it a situation and the questions you want answered, and it answers with types.

Three kinds of question

Install langchain-typesafe, create a key at console.typesafe.ai and put it in .env as TYPESAFE_API_KEY. Then one call carries as many questions as you like, answered together.

from langchain_typesafe import Choice, Noul, Score, TypeSafeClassifier

classifier = TypeSafeClassifier()

answers = classifier.invoke({
    "state": "My card was charged twice for order A17 and nobody has replied.",
    "questions": {
        "urgent": Noul(instructions="Does this need attention today?"),
        "team": Choice(instructions="Which team should take it?",
                       criteria={"billing": "Payments, invoices and refunds.",
                                 "shipping": "Parcels, delivery and returns."}),
        "severity": Score(instructions="How bad is this for the customer?",
                          criteria=["Mild.", "Annoying.", "They have lost money."]),
    },
})

• Noul is a yes or no, and comes back as a probability rather than a word.
• Choice picks one of the options you named, each described in a line, and reports how confident it was.
• Score rates against an ordered list, so "how bad is this" has the same meaning every time.

print(answers.nouls["urgent"].noul)
print(answers.choices["team"].choice, answers.choices["team"].confidence)
print(answers.scores["severity"].score)

No parsing, no prompt asking for JSON, no retry when a model answers "Yes, definitely!" instead of true. The questions are the schema, and the answers arrive shaped.

Where it sits in an agent

This is the middleware slot your guardrail, your mask and your approval step already use. ModelRouterMiddleware puts Jev in front of the model call to decide which model should take the request.

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import ModelChoice, ModelRouterMiddleware

router = ModelRouterMiddleware(
    choices={
        "cheap": ModelChoice(model="openai:gpt-4o-mini",
                             criteria="Order lookups and straightforward questions."),
        "careful": ModelChoice(model="openai:gpt-4o",
                               criteria="Refund disputes and anything a customer has escalated."),
    },
    instructions="Choose the least costly model that can answer safely.",
)

agent = create_agent("openai:gpt-4o-mini", middleware=[router])

Every request now costs one classifier call before the model call it saves, which is the trade: TypeSafe's published figures put that classifier at a small fraction of a model's latency and price. Whether it pays depends on how often your cheap model is enough, which is a number from your own traffic.

AutoModeMiddleware is the other one that ships: given your risky tools, it asks Jev whether a call is safe before it runs, in the same place your approval step sits. Both are marked experimental, and the package is young; the pattern outlives either.

Writing your own

A classifier is only a runnable, so it goes wherever one goes. Holding a TypeSafeClassifier in an AgentMiddleware and classifying in before_agent writes the answer into the agent's state, where the rest of your desk can read it: route the ticket, set its priority, or refuse it before a model is asked anything.

That is four of the five rows swapped. The last one is the saver, and it is the piece that decides whether a customer who comes back tomorrow is still in the conversation.