Timeouts and retries

A timeout is a limit on how long to wait for a call, and a retry is another attempt after a failure; together they keep a program going when the network does not.

Last updated: 30 Sep, 2026 · Python 3.14

Calls over a network fail for reasons outside your code: a busy server, a dropped connection. This lesson uses try and except and async and await to handle both.

Syntax:

await asyncio.wait_for(coroutine, timeout=seconds)   # raises TimeoutError when too slow

Giving up on a call that takes too long

import asyncio

async def stuck_model(text):
    await asyncio.sleep(10)
    return "too late"

async def main():
    try:
        answer = await asyncio.wait_for(stuck_model("ticket 1"), timeout=0.5)
    except asyncio.TimeoutError:
        print("no answer after 0.5 seconds, moving on")

asyncio.run(main())

no answer after 0.5 seconds, moving on

asyncio.wait_for awaits a coroutine but gives up after timeout seconds, cancels it, and raises asyncio.TimeoutError. Without it, one stuck call holds up everything waiting on it.

A model that fails, then works

To practise retries you need a call that fails a set number of times. A class, from Classes, can count its own calls:

import asyncio

class FlakyModel:
    def __init__(self, failures):
        self.failures = failures
        self.calls = 0

    async def ask(self, text):
        self.calls += 1
        await asyncio.sleep(0.1)
        if self.calls <= self.failures:
            raise ConnectionError("the server did not answer")
        return '{"category": "billing", "priority": 4}'

ask is a method written with async def, awaited like any coroutine. It counts every call, and raises until it has failed failures times.

async def main():
    model = FlakyModel(failures=2)
    for _ in range(3):
        try:
            print(await model.ask("refund"))
        except ConnectionError as error:
            print("error:", error)

asyncio.run(main())

error: the server did not answer
error: the server did not answer
{"category": "billing", "priority": 4}

The first two calls raise and the third answers. _ is the usual name for a loop variable you do not use.

Trying again with backoff

async def ask_with_retries(model, text, attempts=3):
    for attempt in range(1, attempts + 1):
        try:
            return await model.ask(text)
        except ConnectionError as error:
            print(f"attempt {attempt} failed: {error}")
            await asyncio.sleep(0.2 * attempt)
    raise ConnectionError(f"gave up after {attempts} attempts")

async def main():
    print(await ask_with_retries(FlakyModel(failures=2), "refund"))

asyncio.run(main())

attempt 1 failed: the server did not answer
attempt 2 failed: the server did not answer
{"category": "billing", "priority": 4}

The return inside the try leaves the function the moment a call works. After a failure it waits a little longer each time, 0.2 then 0.4 seconds, so a busy server gets room to recover. Waiting longer after each failure is called backoff.

Giving up after the last attempt

If every attempt fails, the loop ends and the last line raises, so the caller learns the call did not work instead of receiving None.

async def main():
    await ask_with_retries(FlakyModel(failures=5), "refund")

asyncio.run(main())

attempt 1 failed: the server did not answer
attempt 2 failed: the server did not answer
attempt 3 failed: the server did not answer
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    asyncio.run(main())
  File "main.py", line 2, in main
    await ask_with_retries(FlakyModel(failures=5), "refund")
ConnectionError: gave up after 3 attempts

Timeout vs retry

Where timeouts and retries show up in AI code

• Every model call in production: providers return rate-limit and server errors that clear up after a wait.
• Model SDKs have both built in, as settings such as timeout and max_retries; this lesson shows what those settings do.

Related

• Previous: asyncio.gather
• Next: Virtual environments and pip
• Reference: Timeouts in the asyncio docs