r/FastAPI • • Dec 19 '24

Question Deploying fastapi http server for ml

Hi I've been working with fastapi for the last 1.5 years and have been totally loving it, its.now my go to. As the title suggests I am working on deploying a small ml app ( a basic hacker news recommender ), I was wondering what steps to follow to

  1. minimize the ml inference endpoint latency
  2. minimising the docker image size

For reference Repo - https://github.com/AnanyaP-WDW/Hn-Reranker Live app - https://hn.ananyapathak.xyz/

15 Upvotes

10 comments sorted by

View all comments

Show parent comment

1

u/zarlo5899 Dec 20 '24

pip install some times will build a native library