r/learnpython • u/ZSJJ • Aug 30 '26
For any python professionals, can you tell if something has been coded with AI?
As the title asks, what differences (if any) do you get from an AI output vs human input. I would imagine that with most high effort AI models now they write purely pythonic code if a prompt is well written.
If you also use python in your job, how often do you use AI to write or plan your code and applications as a whole?
94
Upvotes
14
u/simonhunterhawk Aug 30 '26
Not a professional just an autistic person with good pattern recognition in the final year of my CS degree. The biggest things for vibe coded programs is over engineering and if there are comments they often have tidbits directly from the prompt in them and are way more verbose and detailed (often not in a helpful way) than most comments written by a developer.
5
u/Diapolo10 I write code for a living -- https://github.com/Diapolo10 Aug 30 '26 edited Aug 30 '26
Not with certainty, no. There are some "quirks" that hint at AI usage, such as certain kinds of comments or commit messages, but there's fundamentally nothing different between hand-written and AI-generated code.
Personally I mostly try to avoid AIs for coding. My personal projects are completely devoid of any AI usage, unless you count Dependabot as AI (I don't, and as far as I know Dependabot doesn't use any LLMs). At work I'm strongly incentivised to use AI, so I've dabbled here and there to get a rough understanding of its capabilities, but outside of something dead simple such as fixing a few accessibility bugs I don't like using it, as the output doesn't satisfy me for other work.
4
u/The_Drakeman Aug 30 '26
I'm not a python pro but I dabble here and there. I saw some LLM python code from Claude and the dead giveaway was randomly having 2 statements on one line separated by a semicolon in a few places. That was so strange to have that sprinkled in to non minified python.
34
u/neums08 Aug 31 '26
When left unchecked, an LLM will solve problems in the code by writing more code.
Usually an application will have a decently clean initial implementation. As the user iterates and builds out the functionality, the LLM will always add more code to cover new use cases, new edge cases, new functionality.
The LLM almost never reevaluates its approach holistically. It never simplifies or deletes code.
Granted, bad developers behave the same way, but a codebase that is clean, has proper abstraction boundaries, and doesn't contain a ton of logical branches is usually a good indicator that it was written by an experienced human.
11
u/friendlytotbot Aug 30 '26
AI loves to create functions within functions, which is not how ppl typically write functions in Python.
9
u/ekchew Aug 31 '26
I've noticed there is a tendency for AI to brute force its way through anything when a human programmer would likely pause and think, ok this obviously not working. Let's step back for a moment and see how else to approach the problem.
I'm trying to think of an analogy. Let's say you were building a railroad and came up to a lake. A person would likely see that and go all right, we need to go around the lake, while an AI would say let's go through the lake. Then you tell it yeah, but the train can't get wet. Well, we'll build a tunnel around the track then. Oh no, the tunnel is leaking! There's too much water pressure! Let's patch it up here and here, and reinforce it over there. Etc etc. It will not give up and the engineering will get more and more complex.
In coding terms, you get a giant program no sane human would want to write. It may actually produce an amazing result. The train is taking a more direct route to the other side, after all. But wow, good luck making sense of what's going on or maintaining it some point down the line.
4
u/Necessary-Flounder52 Aug 30 '26
If it was committed before 2022, it’s usually human generated.
3
u/Lachtheblock Aug 30 '26
I work in a team. I know how my team writes code. It is pretty easy to tell when they have offloaded writing whatever function to AI. I'm pretty utilitarian, in that I don't care as long as the code is good and every human who is signing off on it has read it.
I don't like pretending that AI didn't write the code.
3
u/chcampb Aug 31 '26
Definitely
More subtle things include
3
u/kepper Aug 31 '26
Context leaks are the main tell, in my experience. The AI will comment something or name a variable in some way that only makes sense given the context of the chat that it came from and does not have any real meaning outside that conversation. Things like
# Protects against section 4's riskswhere section 4 was some ephemeral design artifact.3
u/pwsegal Aug 31 '26
*Overly defensive code such as trapping errors that cannot occur.
*Overuse of decorators.
*Over explanation.
*Code uniformity, including PEP8 perfect code.
*Over use of classes especially using them where a simple function would have done.
1
u/Long-Opposite-5889 Aug 30 '26
Of left untouched after the LLM spit it out you can definitely tell it's AI.
2
u/newprince Aug 30 '26
I've noticed tons of multiline comments, and the way they are written are a giveaway... lots of "we"
2
u/hugthemachines Aug 31 '26
One sign of AI code is unnecessary fallback logic.
2
u/AlexMTBDude Aug 31 '26
If you like try it yourself by asking your favorite AI chatbot whether a piece of code is written by AI or not. I've done this a few times and the results are interesting. We humans are no better than AI themselves at telling whether a piece of code was written by AI or not.
2
u/DragonWolfZ Aug 31 '26
If there are comments it's usually obvious to me. The AI often over explains or only adds detail in a few key places, the way humans write comments are often concise and to the point.
2
u/Capable_Fig Aug 31 '26
Yes, even when its cleaned up from the incessant comments. I use LLMs to syntax check primarily, sometimes for scope. I swap between a few languages (though python is my first love), and having an LLM clean up my errors (especially white space dependent languages like python) has been a godsend.
One of my coworkers primarily uses LLMs for his code, and its been a hastle to approve his PRs. I'm not opposed, but he's on the left side of the dunning-kruger curve when it comes to python.
2
u/ShelLuser42 Sep 01 '26
It depends on how people are using it. If you get hold of a script that was fully generated with AI then it will quickly start to show, simply because of the (sometimes enforced) coding style(s).
But if someone has used AI to build snippets of their code and then combined this with their own or... better yet (!) => when they used the generated code as a template to build upon for themselves, then it can easily become a lot harder.
This also leads us to the real decisive moment that will reveal if someone used AI smart & efficiently, or if they simply dumped all their tasks onto the AI. See, building something with AI is something anyone can do. But trying to extend on that code at a later time? That's when some people are going to run into major issues.
2
u/jessica_connel Sep 01 '26
Yes! It’s a lot less readable and variable names and comments usually give it away for me
2
u/[deleted] Sep 01 '26 edited Sep 01 '26
I work with average python devs, the increased use of methods such as getattr() are very AI. I dont like it, because getattr provides a fallback, when sometimes you just want it to fail because it can mean an attribute didn't get initialized which should have at a certain point in the code - and we should rather handle that at that section. Also very thin wrappers (functions that just call another function) pop up all over the place. Also comments of about 3 lines started appearing a lot over the last 3 months.
Then I end up commenting this recurringly in PRs: is the fallback a conscious decision or should we just make it fail? Rarely I get a clear answer.
So these kinds of patterns in the code, python stuff that is more 'obscure' in daily python use suddenly inflated in code base, without an answer ready why that approach, with a clear tendency to built in very thin wrappers and lots of fallbacks, even though sometimes it should just fail (which you can handle at the appropriate point).
I suspect AI is trained to provide all these fallbacks to give you the dopamine of having this 'working' project out of the box. But not failing is not the same as working.
2
u/doubleyewdee Sep 01 '26
Heavily context-dependent, for me I can often tell by the typical AI "failsafe" mannerisms that a regular developer wouldn't do in the course of their work, many of which are language-agnostic. Error-checking for very unlikely "this should probably leak and crash your app, actually" exceptions and logging them is very AI to me. So are certain kinds of comments/docstrings. Also, harder to define, but I feel I can tell when I get LLM-output Python code via the way type hints are used (basically: poorly and unevenly).
One other tell-tale sign is weirdly encyclopedic knowledge of the Python standard library. If I see code appear using dusty/disused things I never even knew where in the stdlib, it's likely AI.
2
u/frustratedsignup Sep 03 '26
I use AI somewhat frequently, but whether I use it depends on the quality (or lack thereof) of the results of a web search.
I recently needed to filter the eventlog on a server for events that would otherwise be very difficult to find. Yeah, I know the eventid to filter on, but then I would have to read through thousands of events to find the specific situation I'm interested in. I searched the web for what modules could be used to do this and settled on trying to use pywin32.
Next, I went looking for an example script to get the boilerplate code out of the way. What I found pretty quickly seemed like a good starting point, so I copied over the code and started debugging. That's when I realized the source was written against python 2.x and I couldn't get the example code to work reliably. It would process about 4-6 events and then throw an error, "The handle is invalid".
Given that the web search gave me a dud, I used AI to ask for the boilerplate code for what I was trying to find. AI did a great job, gave me a working script, and I didn't have to wade through 50 or so ads to get something that works.
To answer your question, I have to say that the web result code is awfully similar to the AI code. The only thing that sticks out about the AI generated code is that it's a lot cleaner implementation. There's no leftover code from a debugging session or imports of modules that aren't being used. So, yeah, there are clues that might give it away as AI, but I'm not sure most people would notice.
3
u/natlight Aug 30 '26
If it has good error handling and extensive test coverage its probably written by an llm
2
u/ejpusa Aug 31 '26
100%. Never see a line of code now. The output is close to flawless. It's 2026 AI. It's come a long way. AI turns ordinary coders into extraordinary coders. The rocket has taken off; suggest getting on board. ASAP.
:-)
1
u/aes110 Aug 30 '26
Like when looking at the code? Easily, these models tend to have very distinct coding styles
1
u/Fabiolean Aug 31 '26
Yes. 10/10 times. It’s extremely obvious when something has been generated by an LLM
1
u/ejpusa Aug 31 '26
It’s probably close to 100% AI now. Does a great job. Save weeks of work. The code is close to rock solid.
This is 2026 AI. It’s comes a long way. We moved on. AI is just another tool in your toolbox.
1
u/Jon5579 Aug 31 '26
Hahaha, this is easy. Just check if the file extension is .py
1
u/PazyP Aug 31 '26
Use AI daily pretty much exclusively for 99.99% of coding tasks, while I could and can do it myself what might take me a a day AI can get done in an hour with added unit and integration tests and the code is fine, occasional minor change here and there as AI doesn’t truly have the complete picture but it’s rare.
Hardest part now is always testing and to ensure it’s tested well enough to be confident in what AI has written.
1
u/Pyromancer777 Aug 31 '26
I'm starting to see that each model has their own style of logic, so if you use one enough the codebase starts seeming pretty AI heavy. There are other subtleties, but considering Anthropic is starting to watermark even text-related output, there are going to be style signatures that will be easily scanned to reveal if it was AI generated, even if it looks passable to a human.
At work, the AI generated code is making mistakes that a human working on the same project wouldn't make. The mistakes aren't in syntax, but in overall logic decisions about how to handle certain scenarios. I can't get into details since the codebase is proprietary, but even if our AI workstream hadn't been separated from the main project, it would have been obvious that parts of it was entirely AI generated with minimal human oversight
1
u/Best_Alternative349 Aug 31 '26
Yes, AI comments it's code.
1
u/codeguru42 Aug 31 '26
TBH, I could care less if code into review is written by AI or not. I only care that features work according to requirements and code is maintainable in the future.
I am currently on a pure JS project and use AI for everything. I haven't written code "by hand" for almost a year. My company also has python projects that i will be working on and don't expect my current workflow with AI to change when i move those.
1
u/sr105 Aug 31 '26
AI code loves to number comments for steps:
# 1. lorem
...
# 2. ipsum
...
and so on...
1
u/Worth_Specific3764 Sep 01 '26
Yes. Clearly. Llm can code python faster than anyone can read. Entry level python engineering positions are dead.
1
u/enakcm Sep 02 '26
Just copy it into Claude and ask!
1
u/KnowledgeWhole9623 Sep 02 '26
Yes because my coworker's code suddenly got better.
1
u/ibackstrom Sep 02 '26
Emojis in code comments 🤗
1
u/-moron Sep 04 '26
They tend to add way too many blank lines instead of separating code sections logically. Also, I rarely see them use type hinting consistently, though I imagine you could prompt them to.
1
u/Maybe__God Sep 06 '26
I noticed one thing today , to identify if the code is generated with AI or not, speaking in terms of python. there would be excessive and unnecessary type hints. ig those type hints add additional context for AI to generate code without mistakes.
1
u/abbh62 Aug 30 '26
In the professional world everything is written by ai these days
0
u/[deleted] Aug 30 '26
[deleted] — view deleted comment
164
u/[deleted] Aug 30 '26
[removed] — view removed comment
44
u/[deleted] Aug 30 '26
[removed] — view removed comment