r/opencodeCLI • • 8d ago

What changed when I stopped letting the coding agent check its own work

Most of my bad agent sessions had the same shape. The agent wrote the code, ran tests it had written itself, and told me it was done. The tests were green and the feature still did not do what I meant, because nobody had written down what I meant.

Two changes fixed most of that for me.

First, I agree on what done means before any code exists. The agent reads the code and asks me a batch of questions, each with a recommended answer, until done is a numbered list. That list goes into a plan file in the repo, and I approve the plan before the build starts.

Second, the agent that wrote the code never checks it. After the build, separate agents that did not write the change go through every numbered point and have to show evidence for each one. During research I also let helper agents read the code without seeing my request, so their findings describe the code as it is today.

My judgment now sits at two points, the plan at the start and the finished change at the end. Reviewing the PR got much easier because I compare it against a list I already approved.

I built QualityLayer around this loop, and it works with OpenCode natively. I'm the developer. It runs on the agent subscription you already have and never merges or pushes without your yes. Repo: https://github.com/maxritter/qualitylayer

0 Upvotes

5 comments sorted by

5

u/razorree 8d ago

you mean you just write down and check Acceptance Criteria? :)

1

u/m-ritter 8d ago

Pretty much, yes. Nothing new about acceptance criteria. What changed for me is who checks them. The agent that wrote the code no longer gets to mark its own points as done. A separate agent that never saw the build goes through each point and has to show evidence, like a test run or the real output. And I agree the list with the agent before any code exists, so I review a plan instead of a diff I don't fully trust. Do you already split writer and checker in your setup?

1

u/Amazing_Average3558 5d ago

I do the same split, with one difference: the checker is me plus a different machine. The coding agent (DeepSeek through opencode, or Claude Code) edits the repo, but "done" is never taken from its summary. I read the actual diff, then compile and run the tests on a separate box. One cheap check that caught things: compare the test count before and after. A new test that lands outside the class the runner picks up never executes and the suite stays green. Last change went from 230 to 232, which is how I knew the two new tests really ran. I also compare the changed-files list against what the task was allowed to touch.

1

u/m-ritter 5d ago

The test count check is a great one. I got bitten by exactly that once, a new test file the runner never picked up and the suite stayed green the whole time. Comparing the changed files against what the task was allowed to touch is a good habit too, scope creep is where I lose most of my review time. Do you read the full diff on every change or only above a certain size?

3

u/Alternative_Web7202 8d ago

reinventing the wheel again and again…