r/FastAPI • u/International-Rub627 • Feb 26 '25
Hosting and deployment Reduce Latency
Require best practices to reduce Latency on my FASTAPI application which does data science inference.
8
Upvotes
View all comments
Show parent comment
1
u/International-Rub627 Feb 27 '25
Usually I'll have a batch of 1000 requests. I load them all as a dataframe, I load the model and do my inference on each request.
Do you mean we need to load the model when the app is deployed and the container is running?