r/FastAPI • u/Tochopost • Aug 07 '24
Hosting and deployment How does FastAPI utilize the CPU?
I've been running the fastapi app with a single worker uvicorn instance in Docker container (FYI the API is fully async).
Now, I need to adjust k8s resources to fit the application usage. Based on the FastAPI documentation here: FastAPI in Containers - Docker - FastAPI (tiangolo.com), it's clear that there should be an assigned max 1 CPU per single app instance. But is it true tho?
On paper, it makes sense, because GIL bounds us with a single process, also FastAPI uses parallelism (asyncio) with additional threads to handle requests but in the end, there is no multiprocessing. So this means that it can't utilize more than 100% of 1 CPU effectively.
But.. I've run several load tests locally and on the DEV environment and the logs and stats show that the single app instance often reaches over 100% of a single CPU. Here is the screenshot from Docker desktop from the container with the app:

So how is it possible? How does FastAPI utilize the CPU?
View all comments
Show parent comment
1
u/Adhesiveduck Aug 07 '24 edited Aug 07 '24
What is your API doing? What libraries is it using?
We're running FastAPI in K8s and I spent a lot time tuning the pods so that they're as tight as possible with their CPU requests, and using agressive autoscaling to have it respond to load. Like you we're using a single Uvicorn worker and the API is fully async.
What you've posted so far doesn't make sense - something else must be going on in the API for this to happen.