Errors and retries

Provider errors arrive as APICallError. Retryable ones, like rate limits and overloads, are retried with backoff up to maxRetries; the rest fail at once.

Exampleretry.ts

import { generateText, APICallError, RetryError } from "ai";
import { MockLanguageModelV4 } from "ai/test";

let calls = 0;
const overloaded = new MockLanguageModelV4({
  doGenerate: async () => {
    calls++;
    throw new APICallError({ message: "overloaded", url: "https://api.example.com", requestBodyValues: {}, statusCode: 529, isRetryable: true });
  },
});

try {
  await generateText({ model: overloaded, prompt: "Hi", maxRetries: 2 });
} catch (error) {
  console.log(RetryError.isInstance(error), error.reason, calls);
  console.log(error.lastError.statusCode, error.lastError.message);
}

Example

npx tsx retry.ts

The model threw a retryable APICallError with status 529. maxRetries: 2 means three attempts, with exponential backoff between them, and then a RetryError whose reason is maxRetriesExceeded and whose lastError is the provider error. The default is two retries.

Examplebad-request.ts

import { generateText, APICallError } from "ai";
import { MockLanguageModelV4 } from "ai/test";

let calls = 0;
const badRequest = new MockLanguageModelV4({
  doGenerate: async () => {
    calls++;
    throw new APICallError({ message: "invalid model", url: "https://api.example.com", requestBodyValues: {}, statusCode: 400, isRetryable: false });
  },
});

try {
  await generateText({ model: badRequest, prompt: "Hi" });
} catch (error) {
  console.log(APICallError.isInstance(error), error.statusCode, calls);
}

Example

npx tsx bad-request.ts

A 400 is marked not retryable, so it was thrown once, as itself. Retrying a bad request only wastes time. Provider packages set isRetryable from the status code; you check error types with isInstance, which works across package copies where instanceof can fail.

Try it yourself

Set maxRetries: 0 on the first file.
Throw the error only on the first call and let the second succeed.
Catch AI_LoadAPIKeyError from lesson 3 with LoadAPIKeyError.isInstance.

PreviousMiddleware: logging and default settings

Slow is fine. Stopping is the only problem.