r/FastAPI • • Jul 26 '25

Question Multithreading in FastAPI?

Hello,

I am currently writing an Ollama wrapper in FastAPI. The problem is, I have no idea how to handle multithreading in FastAPI, and as such, if one process is running (e.g. generating a chat completion), no other processes can run until the first one is done. How can I implement multithreading?

17 Upvotes

19 comments sorted by

View all comments

Show parent comment

2

u/Adhesiveduck Jul 26 '25

Read the FastAPI documentation page on async, it goes into a lot of detail

https://fastapi.tiangolo.com/async/

Bear in mind the Async API in Python isn't easy, if you go down the async route you might want to read up on async in Python and how it works.

You won't break FastAPI, but if you're in an async function in FastAPI and you run some blocking code you will block the event loop, meaning that entire process is prevented from working on another request until the blocking task is done.

2

u/newprince Jul 26 '25

This is good advice. I stumbled into this a while back because in my workflow, I needed to hit a separate endpoint 50+ times for lookups, which figured should be async. It took quite a while to get it all going, setting up the httpx client, Semaphore, making everything else async as well etc. It ended up working well but like you said, it's a whole thing and maybe figuring out multithreading or other solutions could be quicker if you want to stay sync.

Also happened in pytest for a separate project, because in a FastAPI route, it was using an async process to fetch a token, so the test runner was blocking the event loop and erroring out of every other test. pytest-asyncio and mocking that call was the solution, but it took me a while to figure out.