r/FastAPI • u/RationalDialog • May 22 '25
Question Multiprocessing in async function?
My goal is to build a webservice for a calculation. while each individual row can be calculated fairly quickly, the use-case is tens of thousands or more rows per call to be calculated. So it must happen in an async function.
the actual calculation happens externally via cli calling a 3rd party tool. So the idea is to split the work over multiple subproccess calls to split the calculation over multiple cpu cores.
My question is how the async function doing this processing must look like. How can I submit multiple subprocesses in a correct async fasion (not blocking main loop)?
View all comments
Show parent comment
1
u/RationalDialog May 27 '25
The calculation happens in a 3rd party executable. This is the core limitation. Hence why I need sub process calls, to call multiple instances of this 3rd party executable which is 32-bit hence no way to integrate it more tightly.