I've built most of my backends on FastAPI for the last few years, and I've made this mistake in at least three of them. I've also watched people much smarter than me make it. So here it is, written down, mostly so I stop making it.
The symptom
An analytics API. Local benchmarks look great. Staging looks great. Then it goes live, and the p50 latency is still fine but the p99 is a sawtooth. Random requests take four seconds for no reason. The database is idle. CPU is idle. Nothing is on fire, and yet everything is slow.
It took me longer than I'd like to admit to find it, because the offending line looked completely innocent:
@app.get("/creators/{creator_id}/score")
async def creator_score(creator_id: str):
doc = collection.find_one({"_id": creator_id}) # pymongo. synchronous.
return compute_score(doc)
That's a blocking database call sitting inside an async def. At one request per second in
development, it costs nothing. At two hundred requests per second, that one line stops the
entire server, for everyone, every time it waits on the network.
Why it stops everything
FastAPI runs on an event loop. One thread, one loop, and the deal is: everyone takes turns,
and nobody holds the floor while waiting. When you write async def, you're signing that
deal. You are promising that inside this function, anything that waits will await, so the
loop can go serve other people while you wait.
A synchronous call doesn't await. It just sits there, holding the floor, and every other request in the process queues up behind it. Not slows down. Stops. Until your one database round-trip returns, the server is doing nothing at all.
The counterintuitive part is that plain def would have been safer:
@app.get("/creators/{creator_id}/score")
def creator_score(creator_id: str): # no async
doc = collection.find_one({"_id": creator_id})
return compute_score(doc)
FastAPI runs sync routes in a threadpool automatically. The loop never sees your blocking code. The pool is bounded (forty threads by default), so under heavy load it turns into a queue, but a queue degrades gracefully. A held event loop doesn't degrade. It halts.
The rule
I now apply one rule, and it has removed this class of bug from my life:
If everything that waits inside the function is
awaited, it can beasync def. If even one thing isn't, it'sdef. No exceptions, no "it's just a quick call."
"Everything that waits" includes the sneaky ones: requests.get, time.sleep, open() on a
slow disk, an ORM that isn't async, a logging handler that writes to the network, a
pymongo cursor. If you're not sure, it's def.
And when you genuinely need to call one blocking thing from an otherwise-async function, say so explicitly:
from fastapi.concurrency import run_in_threadpool
async def creator_score(creator_id: str):
doc = await run_in_threadpool(collection.find_one, {"_id": creator_id})
return compute_score(doc)
Finding the ones already in your codebase
The event loop will tell on you, if you ask it to. In development, turn on debug mode and it logs every callback that hogs the loop longer than a threshold:
import asyncio
loop = asyncio.get_running_loop()
loop.set_debug(True)
loop.slow_callback_duration = 0.05 # log anything over 50ms
Run your test suite with that on. Every warning is a sync call wearing an async costume.
When I did this on the analytics service, I found the Mongo call, a PDF generator, and a
"harmless" call to a third-party API. Three loop-stoppers. Each had shipped separately and
each was "too small to matter."
The one thing threads won't fix
CPU work is a different animal. A threadpool helps with waiting, not with computing, because Python's GIL means a thread crunching numbers still starves the loop. If a request needs to score a hundred thousand posts or render a report, it doesn't belong in the web process at all. Hand it to a worker queue, return a job ID, and let the API do what it's good at: being available.
async is a contract, and the runtime trusts you to honor it. Most of my production incidents
in Python have come down to a place where I quietly didn't. Read the fine print.
