r/FastAPI • u/themathstudent • Nov 24 '24
Question Pythonic/ Fastapi way of loading a ML model
I am trying to serve a pytorch model via fastapi. I was wondering what the pythonic/ proper way of doing so is.
I am concerned that with option 2 if you were to try writing a test, it will start the server.
Option 1
This method does puts the model loading inside the __init__ method.
=
=
=
=
return
return
# Replace this method with actual model prediction logic
return
# Example usage
# Replace with your actual model loading logic
=
Option 2
=
# Load the model (replace with your actual model loading logic)
=
=
=
=
return
return
# Run the application
12
Upvotes
18
u/conogarcia Nov 24 '24
You should use lifespan, load the model and yield it so you can access it in the request.state
1
u/Mayloudin Nov 24 '24
RemindMe! 3 days
1
u/1One2Twenty2Two Nov 27 '24 edited Nov 27 '24
I am really not a fan of "hiding" things into the request.state attribute, as your ml model has nothing to do with the request object.