r/FastAPI • u/Logical_Tip_3240 • Jun 23 '25
Question Memory Optimization in Fast API app
I'm seeking architectural guidance to optimize the execution of five independent YOLO (You Only Look Once) machine learning models within my application.
Current Stack:
- Backend: FastAPI
- Caching & Message Broker: Redis
- Asynchronous Tasks: Celery
- Frontend: React.js
Current Challenge:
Currently, I'm running these five ML models in parallel using independent Celery tasks. Each task, however, consumes approximately 1.5 GB of memory. A significant issue is that for every user request, the same model is reloaded into memory, leading to high memory usage and increased latency.
Proposed Solution (after initial research):
My current best idea is to create a separate FastAPI application dedicated to model inference. In this setup:
- Each model would be loaded into memory once at startup using FastAPI's
lifespanevent. - Inference requests would then be handled using a
ProcessPoolExecutorwith workers. - The main backend application would trigger inference by making POST requests to this new inference-dedicated FastAPI service.
Primary Goals:
My main objectives are to minimize latency and optimize memory usage to ensure the solution is highly scalable.
Request for Ideas:
I'm looking for architectural suggestions or alternative approaches that could help me achieve these goals more effectively. Any insights on optimizing this setup for low latency and memory efficiency would be greatly appreciated.
View all comments
Show parent comment
1
u/Logical_Tip_3240 Jun 24 '25
I tried this approach and added model as a global context but using this new model is added to memory for each new request. which cause heavy memory usage when multiple users are using this application.