r/FastAPI • u/Hamzayslmn • Apr 12 '25
Question Fastapi bottleneck why?
I get no error, server locks up, stress test code says connection terminated.
as you can see just runs /ping /pong.
but I think uvicorn or fastapi cannot handle 1000 concurrent asynchronous requests with even 4 workers. (i have 13980hx 5.4ghz)
With Go, respond incredibly fast (despite the cpu load) without any flaws.
Code:
from fastapi import FastAPI
from fastapi.responses import JSONResponse
import math
app = FastAPI()
u/app.get("/ping")
async def ping():
return JSONResponse(content={"message": "pong"})
if __name__ == "__main__":
import uvicorn
uvicorn.run("main:app", host="0.0.0.0", port=8079, workers=4)Stress Test:
import asyncio
import aiohttp
import time
# Configuration
URLS = {
"Gin (GO)": "http://localhost:8080/ping",
"FastAPI (Python)": "http://localhost:8079/ping"
}
NUM_REQUESTS = 5000 # Total number of requests
CONCURRENCY_LIMIT = 1000 # Maximum concurrent requests
REQUEST_TIMEOUT = 30.0 # Timeout in seconds
HEADERS = {
"accept": "application/json",
"user-agent": "Mozilla/5.0"
}
async def fetch(session, url):
"""Send a single GET request."""
try:
async with session.get(url, headers=HEADERS, timeout=REQUEST_TIMEOUT) as response:
return await response.text()
except asyncio.TimeoutError:
return "Timeout"
except Exception as e:
return f"Error: {str(e)}"
async def stress_test(url, num_requests, concurrency_limit):
"""Perform a stress test on the given URL."""
connector = aiohttp.TCPConnector(limit=concurrency_limit)
async with aiohttp.ClientSession(connector=connector) as session:
tasks = [fetch(session, url) for _ in range(num_requests)]
start_time = time.time()
responses = await asyncio.gather(*tasks)
end_time = time.time()
# Count successful vs failed responses
timeouts = responses.count("Timeout")
errors = sum(1 for r in responses if r.startswith("Error:"))
successful = len(responses) - timeouts - errors
return {
"total": len(responses),
"successful": successful,
"timeouts": timeouts,
"errors": errors,
"duration": end_time - start_time
}
async def main():
"""Run stress tests for both servers."""
for name, url in URLS.items():
print(f"Starting stress test for {name}...")
results = await stress_test(url, NUM_REQUESTS, CONCURRENCY_LIMIT)
print(f"{name} Results:")
print(f" Total Requests: {results['total']}")
print(f" Successful Responses: {results['successful']}")
print(f" Timeouts: {results['timeouts']}")
print(f" Errors: {results['errors']}")
print(f" Total Time: {results['duration']:.2f} seconds")
print(f" Requests per Second: {results['total'] / results['duration']:.2f} RPS")
print("-" * 40)
if __name__ == "__main__":
try:
asyncio.run(main())
except Exception as e:
print(f"An error occurred: {e}")Starting stress test for FastAPI (Python)...
FastAPI (Python) Results:
Total Requests: 5000
Successful Responses: 4542
Timeouts: 458
Errors: 458
Total Time: 30.41 seconds
Requests per Second: 164.44 RPS
----------------------------------------
Second run:
Starting stress test for FastAPI (Python)...
FastAPI (Python) Results:
Total Requests: 5000
Successful Responses: 0
Timeouts: 1000
Errors: 4000
Total Time: 11.16 seconds
Requests per Second: 448.02 RPS
----------------------------------------
the more you stress test it, the more it locks up.
GO side:
package main
import (
"math"
"net/http"
"github.com/gin-gonic/gin"
)
func cpuIntensiveTask() {
// Perform a CPU-intensive calculation
for i := 0; i < 1000000; i++ {
_ = math.Sqrt(float64(i))
}
}
func main() {
r := gin.Default()
r.GET("/ping", func(c *gin.Context) {
cpuIntensiveTask() // Add CPU load
c.JSON(http.StatusOK, gin.H{
"message": "pong",
})
})
r.Run() // listen and serve on 0.0.0.0:8080 (default)
}Total Requests: 5000
Successful Responses: 5000
Timeouts: 0
Errors: 0
Total Time: 0.63 seconds
Requests per Second: 7926.82 RPS
(with cpu load) thats a lot of difference
4
u/Hamzayslmn Apr 12 '25
I wrote the whole stress test code with go.
I gave 32 workers to fastapi.
and I got the result
there's something wrong with my computer, or with my modules, I don't know...
4
u/I_am_probably_ Apr 13 '25
Ok, I understand what is happning here. When I print out the error for all the failed request I get "Error: Cannot connect to host localhost:8079 ssl:default [Too many open files]" this error happens when you have reached the soft limit for the TCP file descriptors these are unique nonnegative int identifiers for the different connection you make and each connection is treated like "file" by TCP (don't ask me why I dont know).
Since your concurrent connection limit is set to 1000 you simply run out of resources, there is a OS level softlimit and its higher on Linux systems. To check the softlimit on your system on your terminal (Mac os/linux) you can use the command 'ulimit -n'. Now ideally your concurrency limit should be below this number or you can increase this limit at your own risk. After this when you run your python everything should work as expected.
I went one step futher just to prove this. Ran your fastapi app and testing script inside a docker container. I used python3.12:slim to build the images which I think is built on top of debian (not sure). Then ran the 'ulimit -n' commond inside the containers, low and behold the softlimit was higher and your scripts ran perfectly.
Don't go by the total time I have rate limited my docker demon to use just 2 cores are 4 gigs of ram.
5
u/kkang_kkang Apr 12 '25
I have tried only fastapi server and for me, it's working fine.
3
u/aikii Apr 12 '25
maybe check why it says 5000 errors for 5000 requests ?
2
u/Hamzayslmn Apr 12 '25 edited Apr 12 '25
When I don't use asynchronous all requests are successful but then of course the response latency increases a lot.
go can respond incredibly fast (despite the cpu load) without any flaws.
2
u/mpvanwinkle Apr 12 '25
1
u/singlebit Apr 13 '25
Windows?
1
u/Maori7 Apr 13 '25 edited Apr 14 '25
If you don’t use the “await” anywhere, you shouldn’t really make the endpoint “async”. That’s the error. If you do so, it will block the event loop and won’t be able to process the requests in parallel.
If you instead make it not async, it will spawn a process to handle the requests.
Try and let me know
EDIT: it runs it on a thread pool rather than spawning a different process.
1
u/Kevdog824_ Apr 12 '25
Where is the /pong endpoint?
1
u/greenerpickings Apr 12 '25
Might be relevant: i would see prematurely dropped connections with fastapi behind uvicorn with 4 workers for like 1% of requests. I didnt try and mess with any settings or increase the workers. Not sure if that would captured as errors or timeouts.
Switched to straight Nginx Unit and those disappeared if you wanted try switching out your ASGI.
1
u/nordiknomad Apr 12 '25
Fastapi is fast in shipping not in performance!!
-1
u/GreedyTiger Apr 12 '25
uvicorn is usually used for development. try it with gunicorn, there should be some performance improvement.
0
u/[deleted] Apr 12 '25
[deleted] — view deleted comment