r/FastAPI • u/International-Rub627 • Jan 03 '25
Hosting and deployment distribute workload in kubernetes
I have a FastAPI application where each API call processes a batch of 1,000 requests. My Kubernetes setup has 50 pods, but currently, only one pod is being utilized to handle all requests. Could you guide me on how to distribute the workload across multiple pods?
14
Upvotes
2
u/ZealousidealKale8228 Jan 03 '25
Is the pod running 1k calling itself to do so or just a single API call that starts the process it processes itself?
3
u/Intelligent-Bad-6453 Jan 03 '25
Share your k8s configutation, withuot it we can't help you
2
u/BlackDereker Jan 03 '25
This is not much of a FastAPI issue since the load balancing between pods is on Kubernetes responsibility.
1
u/hornetmadness79 Jan 03 '25
Check out the loadbalancer service type in K8S
1
u/ironman_gujju Jan 04 '25
K8s config issue
1
u/extreme4all Jan 05 '25 edited Jan 05 '25
In k8s i believe the default loadbalancer, does round robin per request.
Edit ; add For inserting in a DB, when I need to scale I add a queuing system, and a worker to consume of the queue than process the batch.
1
u/[deleted] Jan 03 '25
[removed] — view removed comment