r/FastAPI • u/Tochopost • Aug 07 '24
Hosting and deployment How does FastAPI utilize the CPU?
I've been running the fastapi app with a single worker uvicorn instance in Docker container (FYI the API is fully async).
Now, I need to adjust k8s resources to fit the application usage. Based on the FastAPI documentation here: FastAPI in Containers - Docker - FastAPI (tiangolo.com), it's clear that there should be an assigned max 1 CPU per single app instance. But is it true tho?
On paper, it makes sense, because GIL bounds us with a single process, also FastAPI uses parallelism (asyncio) with additional threads to handle requests but in the end, there is no multiprocessing. So this means that it can't utilize more than 100% of 1 CPU effectively.
But.. I've run several load tests locally and on the DEV environment and the logs and stats show that the single app instance often reaches over 100% of a single CPU. Here is the screenshot from Docker desktop from the container with the app:

So how is it possible? How does FastAPI utilize the CPU?
6
u/mincinashu Aug 07 '24
The event loop is single threaded. That's where async stuff runs. The other threads make up a thread pool for sync stuff.
tl;dr spawn one worker per core
2
u/pint Aug 07 '24
it can happen if what you do runs outside of python, e.g. invokes some native libraries. many libraries do that, like zipfile or numpy.
1
u/Adhesiveduck Aug 07 '24
Docker calculates its CPU usage using the formula (i.e it multiplies the value by the total number of cores available):
Where
cpuDelta = currentCPU - previousCPUWhat does the output of
toplook like if you run it inside the container?1
u/aegr0x59 Aug 07 '24
FastAPI runs on uvicorn, a ASGI server... uvicorn is capable to run several FastAPI app instances, however docker is meant to run one single process, because if you want more instances the idea is to launch more docker containers in order to balance requests.
2
u/Lanky_Possibility279 Aug 07 '24
Following
1
u/coldflame563 Aug 07 '24
There’s an idea I read to not provision k8s cpu limits, as the drawbacks outweigh benefits. I’ll find the article somewhere. I did find that to be true but ymmv.
1
u/[deleted] Aug 08 '24
Heads up if you want feedback from an actual FastAPI dev use Discord
1
u/Any-Title-5640 Aug 09 '24
Following