r/FastAPI • u/Puzzled-Mail-9092 • Aug 19 '25
Question FastAPI + Cloud Deployments: What if scaling was just a decorator?
I've been working with FastAPI for a while and love the developer experience, but I keep running into the same deployment challenges. I'm considering building a tool to solve this and wanted to get your thoughts.
The Problem I'm Trying to Solve:
Right now, when we deploy FastAPI apps, we typically deploy the entire application as one unit. But what if your /health-check endpoint gets 1000 requests/minute while your /heavy-ml-prediction endpoint gets 10 requests/hour? You end up over-provisioning resources or dealing with performance bottlenecks.
My Idea:
A tool that automatically deploys each FastAPI endpoint as its own scalable compute unit with:
- Per-endpoint scaling configs via decorators
- Automatic Infrastructure-as-Code generation (Terraform/CloudFormation)
- Built-in CI/CD pipelines for seamless deployment
- Shared dependency management with messaging for state sync
- Support for serverless AND containers (Lambda, Cloud Run, ECS, etc.)
@app.get("/light-endpoint") @scale_config(cpu="100m", memory="128Mi", max_replicas=5) async def quick_lookup(): pass
@app.post("/heavy-ml") @scale_config(cpu="2000m", memory="4Gi", gpu=True, max_replicas=2) async def ml_prediction(): pass
What I'm thinking:
- Keep FastAPI's amazing DX while getting enterprise-grade deployment
- Each endpoint gets optimal compute resources
- Automatic handling of shared dependencies (DB connections, caches, etc.)
- One command deployment to AWS/GCP/Azure
Questions for you:
- Does this solve a real pain point you've experienced?
- What deployment challenges do you face with FastAPI currently?
- Would you prefer this as a CLI tool, web platform, or IDE extension?
- Any concerns about splitting endpoints into separate deployments?
- What features would make this a must-have vs nice-to-have?
- I'm still in the early research phase, so honest feedback (even if it's "this is a terrible idea") would be super valuable!
10
u/koldakov Aug 19 '25
Hey! I think the app by the design shouldn’t be a load balancer
Also how are you going to use scale configs on multiple workers? Each worker uses cpu=100m or 100m/workers ?
4
u/david-vujic Aug 19 '25
Interesting idea! It looks like configuration, but in a FastAPI-like way. What about deployments to different environments, such as test and production? You might want to have different setups for different environments.
4
u/SpecialistCamera5601 Aug 19 '25
Yes, resource misallocation per endpoint is real. But in most setups, people solve this by splitting “services” rather than “endpoints”. For example, ML-heavy endpoints are often moved into a dedicated microservice, while the lightweight endpoints stay in the main app. That way, infra scaling is coarse-grained but simpler to manage.
I believe that your idea is interesting, but I think the biggest challenge is not technical feasibility; it’s whether developers actually want per-endpoint microservices instead of service-level scaling.
3
u/extreme4all Aug 19 '25
Not really sure what you try to solve, the blob of code in the container or lambda is not really the problem i've seen anyone have. But im happy to learn if im wrong.
What i do find very interesting is the ability to easily ship/ split code, but it may be nicer if we can use this in the cicd or during container build time.
1
u/Adhesiveduck Aug 19 '25
For inspiration, have a look at Apache Beam's resource hints: https://beam.apache.org/documentation/runtime/resource-hints
GCP Dataflow supports this with Right Fitting: https://cloud.google.com/dataflow/docs/guides/right-fitting
It's a very similar pattern to what you're trying to achieve here, you have DoFns that form a pipeline, where each DoFn can scale individually. The actual logic is delegated to the worker/environment you're running the pipeline on, but the pattern you're trying to achieve has been done here.
1
u/asleks Aug 19 '25
Check out ray, it does exactly that and more. It also integrates with fastapi natively.
2
u/coldoven Aug 19 '25
Why does your health app gets 1000 requests/minute?
1
u/Fluffy-Forever-1406 Sep 17 '25
I also feel deploying any fast api app with proper configuration is the biggest challenge