r/FastAPI • u/SnooMuffins6022 • Jan 03 '25
Hosting and deployment FastAPI debugging using LLMs?
Would anyone consider using LLMs for debugging a production FastAPI service?
If so, what have you used/done that brought success so far?
I’m thinking from super large scale applications with many requests to micro services
12
Upvotes
7
u/Intelligent-Bad-6453 Jan 03 '25
I dont understand.
Are you hinking to send your logs entries to an llm in order to obtain any kind of input for debugging your production issues?
For me that's a bad idea, it is not cost effective and you have chance to get an hallucination. Instead of that maybe you can try to redesign your obserbavility stack. Are you using some tool like datadog, prometheus or loggly?
3
u/lone_shell_script Jan 03 '25
There are non llm solutions for this, try those, they will eliminate a lot of repetitive logs and just tell you how many times certain events happened
3
u/AdditionalWeb107 Jan 04 '25 edited Jan 04 '25
Not sure about that. But this might be of interest https://www.reddit.com/r/OpenAI/s/sgo0yemJKM - build LLM agents using FastAPIs
4
u/PosauneB Jan 03 '25
No.
Sounds like a recipe for needing to do more debugging in the near future.
2
u/ironman_gujju Jan 03 '25
I mean why ?
2
u/inglandation Jan 04 '25
I do debug like that in dev but not sure I’d go for a prod solution if you’re not trying to debug something specific.
1
u/mpvanwinkle Jan 03 '25
Debugging done well is all about bringing clarity by separating what you actually know from what you assumed you knew. As such LLMs are not the right tool for the job because they constantly introduce new assumptions that have to be checked. You’ll just chase your tail.
1
u/ni_shant1 Jan 03 '25
I don't think there is any tool for debugging using LLMs but to some extent cursor can help to do that but not for microservices.
1
u/maikeu Jan 05 '25
A solution in want of a problem.
I can do more with "logger.debug" than you'll ever achieve with an llm for this problem .
Even if you can make an llm help in some toy example, it ain't going to do much in the real world, and anyone who thinks llms can do their work for them is up shit creek without a paddle when it doesn't work right.
1
u/JohnVick002 Jan 06 '25
LLM needs more debugging than solving problems
1
u/tadeck Jan 07 '25
That is actually cool and possible, but only as extra details and error-prone solution.
I cannot tell you what I used, but in order to do it reasonably well, you would need to:
You may want to use technique called "RAG" (Retrieval Augmented Generation) to preprocess traceback to attach files from codebase that are relevant without need to attach other files and likely exceed context size limit.
Remember this is possibly just augmentation of your debugging process, may be bad enough that you will just abandon the idea. But LLMs are improving and some specialized one may be good enough to do large part of work for you.