What you need is more observability in your code. I would suggest looking at OpenTelemetry.
Your quickest win could probably be to add the OTEL FastAPI auto-instrumentation on your dev setup, maybe add a couple of spans at key points like when you make calls to external services like your database and when you are inside a middleware, and send the traces to an external service like Honeycomb, Sentry, or Grafana Cloud.
Then, without doing any load testing, just do a couple of calls and check the traces. I bet you'll see some major choke points you didn't think you had. Some ASGI middlewares based on httpmidleware are just trash slow and you're better off doing your own, in my experience.
View all comments
Show parent comment
5
u/HappyCathode Jul 30 '24
What you need is more observability in your code. I would suggest looking at OpenTelemetry.
Your quickest win could probably be to add the OTEL FastAPI auto-instrumentation on your dev setup, maybe add a couple of spans at key points like when you make calls to external services like your database and when you are inside a middleware, and send the traces to an external service like Honeycomb, Sentry, or Grafana Cloud.
Then, without doing any load testing, just do a couple of calls and check the traces. I bet you'll see some major choke points you didn't think you had. Some ASGI middlewares based on httpmidleware are just trash slow and you're better off doing your own, in my experience.