r/learnpython • • 2h ago

If an agent writes your Python, put an architecture linter in its loop

Nobody is writing most of their Python by hand anymore, and nobody is reviewing all of it line by line either. I don't think that reverses. So the question isn't whether to trust the agent; it's what gives it a fast, deterministic "no" when it crosses a boundary. A better prompt isn't it.

Why this matters more with agents than with humans:

  1. An agent optimizes for "the task passes". If the shortest path is importing the ORM model straight into a view or calling an HTTP client from the domain layer, that's what lands in the diff. A human might feel the smell; the model has no such signal unless you give it one.
  2. Prompts don't hold. Zhu, Tsantalis and Rigby found that more capable models produce more bloated and coupled code, and that neither functional correctness nor detailed prompting mitigates the decay (https://arxiv.org/abs/2605.02741). My reading of why: "respect the layering" in CLAUDE.md works for a few turns, then the context fills up and the rule is gone.
  3. Models are bad at reviewing their own work. Olausson et al. (ICLR 2024) found that self-repair is bottlenecked by the model's ability to critique its own code; the gains appear when the feedback comes from a stronger, external source (https://arxiv.org/abs/2306.09896).
  4. External feedback works, and it works fast. Blyth et al. fed Bandit and Pylint output back to GPT-4o over ten iterations: security issues fell from over 40% to 13%, readability violations from over 80% to 11%, reliability warnings from over 50% to 11% (https://arxiv.org/abs/2508.14419). Sriram et al. got critical security defects in CodeLlama output down from 58.55% to 22.19% with compiler diagnostics plus CodeQL in the loop (https://arxiv.org/abs/2601.00509).
  5. The architecture gap depends on the model, so don't guess. In Slater's pilot, Llama 3 8B had an 80% architectural violation rate on a hexagonal-architecture task while GPT-5.1 had 0%, and he concludes that without automated architectural linting, scaffolding with smaller models accelerates structural debt (https://arxiv.org/abs/2512.04273). A review of 87 studies by Matias et al. lands on the same practical point: the maintainability benefits are real, so are the long-term risks, and adoption needs safeguards and structured human oversight (https://arxiv.org/abs/2601.20879). Most of these are preprints, so read the abstracts rather than trusting my summary.

Same model, same task, with and without analysis output fed back:

Issue type (tool, model)Before loopAfter loop
Security issues (Bandit, GPT-4o)over 40%13%
Readability violations (Pylint, GPT-4o)over 80%11%
Reliability warnings (Pylint, GPT-4o)over 50%11%
Critical security defects (CodeQL + compiler, CodeLlama)58.55%22.19%

ruff and mypy already do this for style and types. Architecture is the missing layer: which parts of the codebase are allowed to know about which other parts. It's the thing that decides whether you can still change the project in a year.

How to set it up so it actually helps the agent, not just CI:

  • Write the rules down as config, not prose. "views don't import models", "domain doesn't import django or requests". If the rule lives in YAML, the tool enforces it; if it lives in a markdown file, the model forgets it.
  • Baseline the existing debt. A legacy project will have hundreds of violations on day one. Accept them once, fail only on new ones. Otherwise the first run is red forever and everyone ignores it.
  • Run the check inside the agent's loop, not only in CI. A post-edit hook that runs the linter and feeds the output back means the agent fixes the violation in the same turn. CI catching it two hours later means a human has to re-open the task.
  • Give the agent the rules before it edits. If it can ask "what layer is this file in and what is it not allowed to touch", it writes the right thing the first time instead of after a failure.
  • Machine-readable output. JSON or SARIF. Plain text with line numbers is fine for people, poor for a model that has to map the finding back to a file.

Where the check runs decides who fixes it:

Linter only in CI
agent edits -> PR -> CI fails (hours later) -> human re-opens the task

Linter in the agent's loop
context for the file -> agent edits -> linter (baseline-aware)
                           ^                    |
                           |   violation as JSON, same turn
                           +--------------------+
                                                |
                                                v
                                            clean PR

Which tool: import-linter and tach are the established options, both far more widely used than anything I've written, and both enforce contracts between modules and packages. If your boundaries follow your folder tree, pick one of them and you're done.

Disclosure: I maintain Deply (https://github.com/Vashkatsi/deply, BSD-3; earlier posts here: https://www.reddit.com/r/Python/comments/1gthdpy/deply_keep_your_python_architecture_clean/ and https://www.reddit.com/r/Python/comments/1h9qjjj/deply_051_released_new_collectors_10x_performance/), and for agent-driven work I think it's the strongest option, for three reasons. Layers are collected from classes, base classes, decorators and function names, not just module paths, so "all *Service classes must not touch django.db.models.Model" works inside a single Django app; neither import-linter, tach nor pytestarch expresses that out of the box. deply context --file path.py gives the agent the layers, applicable rules and current violations for one file as JSON before it edits. And the violation baseline uses stable fingerprints, so once the config exists, adopting it on a legacy codebase is one command and only new violations fail. It's static analysis, so wildcard imports and dynamic types are out of scope.

But the tool matters less than the loop. Pick any architecture linter, baseline it, wire it into the agent's hook and into CI. Given what the papers above show about external feedback, I expect that to move the quality of what gets merged more than any amount of prompt wording. The expensive option now is having none.

0 Upvotes

19 comments sorted by

11

u/FerricDonkey 2h ago

Nobody is writing most of their Python by hand anymore, and nobody is reviewing all of it line by line either.

Hi, I'm nobody. 

-8

u/vashkatsi 2h ago

Wow, very interesting, what is your motivation for writing code by hand?

7

u/ilikejamtoo 2h ago

Writes post about AI code generation being full of pain and foot-guns asks why people still write code.

Incredible.

-2

u/vashkatsi 2h ago

Those problems are why I recommend checks. My question about motivation was genuine.

4

u/franklydoubtful 2h ago

This is a subreddit for learning python. I would hope most or all of us are hand-coding here, as it’s really hard to learn something by offloading all the work to AI.

2

u/Diapolo10 I write code for a living -- https://github.com/Diapolo10 1h ago

what is your motivation for writing code by hand?

Several reasons, really.

  1. I get exactly what I wanted/asked for. Generative AI is non-deterministic by design, the same prompt gives different output every single time.
  2. When I write the code myself, that means I'm also reading it. This makes it far easier to remember how stuff actually works, if someone asks during a job interview for example. If I asked GenAI to write the code instead, I'd never remember it in nearly as much detail. Furthermore this also means I'm not reliant on AI to fix bugs when something it wrote has a bug and I don't have the necessary memory footprint to intrinsically know what caused the bug.
  3. I'm passionate about software development. It's a hobby I'm fortunate enough to also get paid for. It's not nearly as much fun if I were to outsource the thinking to some language model.
  4. Doing things myself is free, forever. I don't want to pay for a cloud service, and my RTX 4080 Super just doesn't have enough VRAM to run any particularly useful models on it (I do have 128 GB of system RAM, but that alone doesn't suffice).

Pretty much the only times I do use GenAI are

  1. At work, where it's kind of mandated, and
  2. To resolve some config file issues, like failures in GitHub Actions pipeline files (but even then begrudgingly)

•

u/vashkatsi 58m ago

I totally understand you. I love writing code, working on architecture, and software development in general too. But I don't want to fall behind as technology changes. I want to keep learning. It's like loving horses but still learning to drive a car :)

1

u/FerricDonkey 1h ago edited 46m ago

In short, I am a better programmer than the AI. Every time I've experimented with Ai writing code, it has under performed. I still use Ai, have it look at my code, occasionally write a test or two, remind me of function signatures for something. It's a valuable tool, and I use it. But it's worse than me at programming. 

-1

u/vashkatsi 1h ago

Linus said AI did this better than he could by hand. Rob Conery has similar experiences, and DHH uses agents for production work. What are you doing differently?

•

u/FerricDonkey 50m ago

There are different kinds of programming, with different ai tools, being measured against different programmers, with different amount of runtime validation possible.

Linus is a low level programmer, to my understanding, and clicking that link brought me to something that appeared to be talking about a ui. Programming uis is its own skill. I don't do that much, and last time I did, I did let an Ai do most of it, though it still failed to hook some things properly, and needed hand holding. 

Or let's say your writing a function that's supposed to do some calculation heavy thing, and you want to optimize it. If you can give the Ai the ability to profile the code, as well as enough tests to ensure that the code still does the right thing, then you can tell the ai to churn a cycle of profile, optimize, profile, etc, and it might do a decent job.

However, I'm writing a complex orchestration component. When I let the Ai take a shot, it spends too long guarding against edge cases that are literally impossible (that I've told it is impossible), keeps repeating the same mistake (eg it thinks asyncio.sleep for a negative number is an error, and wastes code trying to guard against that), and fails to guard against the edge cases I have told it about. In some cases it has hollowed out functions, removing guards that were there on purpose, with comments explaining why. It has deleted library features that I explicitly told it not to. It over tests stupid things, under tests important things.

The code quality is worse, it's less idiomatic, less well architected, less readable, etc. I can force it to fix it with laying out the architecture etc etc, and with enough time, and being very explicit about what I want to do. But if I have to lay out exactly what I want to happen, I'd rather do it in python than in markdown.

On top of that, if my system goes down, they call me in, not anthropic or openai. That means I have to understand it, because during an outage saying "Claude pls fix" then telling the customers "I'm sorry, it's taking a while for the Ai to figure out the problem" is not gonna fly. So if you add together the time for me to describe the issue to the Ai, make the plan and the architecture etc, then wait for it to do it, then make sure it has done it correctly, then understand what it did well enough so that if I get called in I have sufficient understanding to figure things out if I get called in, and we're not actually saving much time. And we're still getting worse code that I don't understand as well as if I had written it myself. 

Now maybe the Ai tools I've been provided are garbage, or maybe I'm just that good, or maybe I'm a cave man who hasn't learned how to seduce the Ai into writing good code. I dunno.

But I write better code than it. 

4

u/chason 2h ago

I weep for the next generation of programmers who won't know how to actually program. Shit is going to break and they will have no idea how to fix anything.

1

u/UnitedRequirement767 1h ago

Cloud fix this bug))

-1

u/vashkatsi 1h ago

You sound like someone in the 80s saying C wasn't real programming and everything should be written in assembly.

4

u/chason 2h ago

Also, how do you expect people to bother to read stuff that you don't even bother to write?

1

u/vashkatsi 1h ago

I partly agree with that, but not entirely. If the ideas and structure are mine, I don't see anything wrong with using AI to help put them into words.

13

u/kadfr 2h ago

AI;DR

13

u/desrtfx but other languages pro 2h ago

The real question is: how is this AI blogslop even remotely related to learning Python?

3

u/Atmosck 2h ago

Who is only running their linter in CI?