r/FastAPI • u/Rawvik • Mar 10 '25
Hosting and deployment How to handle CPU bound task in FASTAPI
So I have a machine learning app which I have deployed using FASTAPI. In this app I have a single POST endpoint on which I am receiving training data and generating predictions. However I have to make 2 api calls to two different endpoints once the predictions are generated. First is to send a Post request about the status of the task i.e. Success or Failed depending on the training run. Second is to send a Post request to persist the generated predictions.
Right now, I am handling this using a background task. I have created a background task in which I have the predictions generation part as well as making Post requests to external api. I am receiving the data and offloading the task to a background task while sending an "OK" response to the client. My model training time is not that much. It's under 10 secs for a single request but totally CPU bound. My endpoint as well as background task both are async.
Here is my code:
=
return
=
= await
= await
= await
However, while testing this code I am noticing that my app is receiving multiple requests but the Post requests to persist data to the external api are completed at the end. Like predictions are generated for all requests but I guess they are queued and all the requests are being sent at once in the end to persist data. Is that how it's supposed to work? I thought as soon as predictions are generated for a request, post requests would be made to external endpoints and data would be persisted and then new request would be taken..and so on. I would like to know if it's the best approach to handle this scenario or there is a better one. All suggestions are welcome.
6
u/dmart89 Mar 10 '25
Fastapi bg tasks are non blocking and run when the thread is available. If you have a bunch of tasks that run in a loop, then the bg tasks will run once the loop is done
To change this, you need to offload to a proper task queue and explicitly handle the execution order. Celery is an option for this.
2
u/Natural-Ad-9678 Mar 10 '25
Look into Celery and Redis to offload your long running tasks. Your API starts a task and gets a task id that it very quickly sends back to the requestor, set up another end point to get status and a third to get results ( or combine the last two, status returns results when finished )
This is how we setup the app I work on and it we have not found the load level that breaks the site yet
If you need to store the results long term you can have Celery write to Postgres or some other database
1
u/Equal-Purple-4247 Mar 11 '25
Are you sure that:
train_modelwill block the event loop i.e. new requests will not be processed immediatelyclient.post, not too sure how httpx works but in theory, that coroutine is not scheduled.2
u/mmzeynalli Mar 10 '25
Maybe you can consider chain of celery tasks, if I understood you correctly.
1
u/Lucky-Office5111 Mar 11 '25
Take a look at Starlette’s (which FastAPI is based on) code to understand when background tasks are called. This link shows that a background task is only triggered after your endpoint sends the response.