Real models: providers, keys and switching

dspy.LM takes a LiteLLM model string, so one line picks OpenAI, Anthropic or a local model. dspy.context switches the model for part of a program.

The part before the slash is LiteLLM's provider name, and the key is read from that provider's environment variable. dspy.LM also takes temperature, max_tokens and cache, used on every call.

export OPENAI_API_KEY="sk-..."

Switching models for one block

mock = dspy.LM("openai/gpt-4o-mini", mock_response="[[ ## category ## ]]\nshipping\n\n[[ ## completed ## ]]")
sort = dspy.Predict(Triage)

print(sort(ticket="I was charged twice").category)
with dspy.context(lm=mock):
    print(sort(ticket="I was charged twice").category)
print(sort(ticket="I was charged twice").category)

dspy.configure sets the model for the whole program. dspy.context(lm=...) overrides it only inside the with block, and every module called there uses it. The mock answered shipping inside the block, and the stand-in answered outside it.

sort = dspy.Predict(Triage)
sort.set_lm(dspy.LM("openai/gpt-4o-mini", mock_response="[[ ## category ## ]]\naccount\n\n[[ ## completed ## ]]"))
print(sort(ticket="I was charged twice").category)
print(sort.get_lm().model)

set_lm pins a model to one module, whatever is configured globally. A pipeline can use a cheap model for sorting and a stronger one for writing replies.