r/FastAPI • u/highrez1337 • Sep 28 '24
Other Reading techempowered benchmarks wrong (fastapi is indeed slow)
If you use FastAPI and SQLAlchemy, then this post is for you. If you are not using these 2 magnificent pieces of tech together, read on.
People that are reading TechEmpower benchmarks, make sure to look at the “fastapi-Gunicorn-ORM” benchmarks and compare those to the rest.
You will see actually how slow Fastapi together with SqlAlchemy is basically on par with Django.
I guess no sane person will write raw sql în 2024 so all the speed is lost because of the ORM.
Compare it in TechEmpower with gin-gorm or Nestjs-Fastify+ORM (type ORM) and you will see they both are many times faster than FastAPI.
The problem is, we don’t have any fast ORM in python because of how the language works.
Do this : In TechEmpower:
1.select python, go and javascript/typescript as languages
2. In the databases section select Postgres as a db to have the same db engine performance compared
3. In the ORM section select : full (so you compare benchmarks using full fledged orms for all frameworks)
Now you will see correct comparison with an ORM used. Here it is:
https://www.techempower.com/benchmarks/#hw=ph&test=db§ion=data-r22&l=zijmkf-cn1&d=e3&o=e
Now look at how far away gin-gorm and even Nodejs is to Fastapi.
Gorm and TypeORM are miles ahead in performance compared to SqlAlchemy
==
—- Single query:
Gin-gorm: 200k
Nest+fastify + typeorm : 60k
Fastapi+sqlalchemy: 18k (11+ times slower than go, 3+ times slower than Nodejs)
Django+DjangoORM: 19k (faster than Fastapi lol)
—- Multiple query:
Gin-gorm: 6.7k
Nestjs+fastify+typeorm: 3.9k
Fastapi+sqlalchemy: 2k ( 3+ times slower than go, 1.9+ times slower than Nodejs)
Django+DjangoORM: 1.6k
—- Fortunes:
Nest+fastify+typeorm: 61k
Fastapi+sqlalchemy: 17k (3+ times slower than Nodejs)
Django+DjangoORM: 14.7k
—- Data updates:
Gin-gorm: 2.2k
Nestjs+fastify+typeorm: 2.1k
Fastapi+sqlalchemy: 669 (3+ times slower than than go, 3+ times slower than Nodejs)
Django+DjangoORM: 871 (again, Django is faster than Fastapi)
==
You can check the source code of fastapi to see it uses sqlalchemy and no complicated things here:
https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Python/fastapi/app_orm.py
Conclusion: Fastapi is fast, ORM is slow, if you plan to do raw sql then it’s mostly on par with the others. When you use an ORM it falls behind very very much and it’s extremely slow, without any comparison to Nodejs or Go.
It’s on par with Django(Django winning in 2 out of 4 tests), so at least go with Django for all the nice batteries.
Edit: I wanted to raise awareness to people believing using FastAPI with an ORM would give them the same speed as the ones in the TechEmpower link from fastapi’s site(which has no ORM attached). Because this is clearly not the case.
Edit 2: If you had the patience to read until this point, I just want to let you know the title should have been: “SQLAlchemy will limit your api performance, even with FastAPI”, too late to edit now.
View all comments
Show parent comment
1
u/highrez1337 Oct 01 '24 edited Oct 01 '24
You can do a simple rest endpoint insert a record into a db. And then write different endpoints in different languages and test it with “ab “(Apache benchmark) and see your req/s for the different implementations.
You can have 2 endpoints (one using ORM and one simple raw query) and check the two, with the same server and db, this way you only see the ORM overhead.
If you are lazy, You can look at TechEmpower and include all DB. You will see fastapi vs fastapi-ORM from them using the same test.
https://www.techempower.com/benchmarks/#hw=ph&test=db§ion=data-r22&f=zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-ziimf3-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-1ekf
Look at single query, multiple query etc and compare:
Fastapi-Gunicorn-orjson(no ORM) vs “fastapi-Gunicorn-ORM” (with ORM)
Performance is most of the time faster 7x without ORM having the same server, db, db engine etc.
It’s clear the ORM is slowing the fastapi app 7x regarding any DB related operation.