async and await: code that waits

A real model takes a second or more to answer, and nearly all of that time your program is waiting on the network. This lesson measures that wait.

import time

def ask_model_slowly(text):
    time.sleep(0.5)
    return f"answer to: {text}"

start = time.perf_counter()
for text in ["ticket 1", "ticket 2", "ticket 3"]:
    print(ask_model_slowly(text))
print(f"took {time.perf_counter() - start:.1f} seconds")

time.sleep(0.5) pauses for half a second, standing in for the network. time.perf_counter reads a clock, and :.1f in the f-string shows one digit after the point.

Three calls, half a second each, one after the other: a second and a half. A hundred tickets would take fifty seconds, nearly all of it waiting.

The async version

import asyncio

async def ask_model_slowly(text):
    await asyncio.sleep(0.5)
    return f"answer to: {text}"

async def main():
    for text in ["ticket 1", "ticket 2", "ticket 3"]:
        print(await ask_model_slowly(text))

asyncio.run(main())

async def makes a coroutine function: calling it does not run it straight away. await runs it and waits for the result, and while it waits, Python is free to work on something else.

await only works inside an async def. asyncio.run(main()) is the one line that starts the async part of a program from ordinary code.

This version is not faster yet. It still awaits one ticket before starting the next. What changed is that the waiting can now be shared, which is the next lesson.

Forgetting await

async def main():
    answer = ask_model_slowly("ticket 1")
    print(answer)

asyncio.run(main())

No answer, no error: just a coroutine object, the work that was never started. Python also prints a RuntimeWarning: coroutine ... was never awaited to the error stream. When an async function seems to return something strange, look for a missing await first.