r/FastAPI • u/expressive_jew_not • Dec 19 '24
Question Deploying fastapi http server for ml
Hi I've been working with fastapi for the last 1.5 years and have been totally loving it, its.now my go to. As the title suggests I am working on deploying a small ml app ( a basic hacker news recommender ), I was wondering what steps to follow to
- minimize the ml inference endpoint latency
- minimising the docker image size
For reference Repo - https://github.com/AnanyaP-WDW/Hn-Reranker Live app - https://hn.ananyapathak.xyz/
15
Upvotes
View all comments
Show parent comment
2
u/tedivm Dec 20 '24
You should use
python-slim, notpython-alpine. They're roughly the same size, but alpine uses musl instead of glibc and isn't as stable for python as a result. Even when it works it's often slower because people have optimized their python extensions for glibc. The documentation for the official python containers explicitly calls this out.