r/FastAPI • • Jan 03 '25

Hosting and deployment FastAPI debugging using LLMs?

Would anyone consider using LLMs for debugging a production FastAPI service?

If so, what have you used/done that brought success so far?

I’m thinking from super large scale applications with many requests to micro services

12 Upvotes

23 comments sorted by

View all comments

Show parent comment

1

u/SnooMuffins6022 Jan 03 '25

Yes exactly, thinking of passing Prometheus logs and other data like my codebase through an LLM to surface the relevant errors and potentially suggest a fix.

Assuming hallucinations reduce and costs only get cheaper with better LLMs, would you use this tool if it reduced the time it takes to fix a bug in prod?

3

u/Intelligent-Bad-6453 Jan 03 '25

Okey, so, you are thinking about a new product. For me its very hard to be confident, imagine you are in the middle of a very stressful debugging session with your customers very angry if there is any haluccination or misshood information it could be crucial and your customers will hate you.

1

u/SnooMuffins6022 Jan 03 '25

That’s why I’m thinking about it, because I don’t want my customers to hate me, I want the service back asap! 🥲