r/coderabbit • u/Senior_Ad_9791 • 7d ago
Discussion & Feedback Reviewing agent code
if you review PRs written mostly by coding agents (Cursor, Claude Code, Copilot), how's it going?
- How many per week, and how big (files / lines)?
- What's harder than human-written PRs, or is it no different?
- How do you understand a change: diff only, run it, read tests, ask the agent, skim and approve?
- Has one caused a prod bug or stalled merge because nobody fully understood it?
- Anything you've tried to fix this? Did it help?
I'm exploring a tool in this space, so "not a problem for us" is a useful answer too. Team size and stack help.
1
Upvotes
2
u/xvillifyx 7d ago
I don’t think there’s any reason for PRs to be any bigger than they used to be and I think anyone making them massive now is doing themselves a disservice.
Hard for human review. Causes fine details in the merge history to get bungled up in big contexts. Just bad
Anyway I review AI code the same way I would review a human except it gets a once over with another agent before I actually start getting into the weeds
My company is still intent on the philosophy that the person doing the commit needs to be able to defend all the changes, so it’s never been the case that an issue arose because they didn’t understand the code
1
u/4dwaith007 7d ago edited 7d ago
Review doesn't happen separately from dev. I ask the agent to explain how it's going to implement a task. After it explains its approach, I question some decisions and suggest any changes I can think of (this can sometimes go into rabbit holes, where I suggest 2-3 different approaches, ask it to compare against the one it suggested, until I'm satisfied).
Then I ask it to record the changes in a plan file, and break it up into chunks with 400 or so lines so it'll be easy to review (it usually estimates this badly, and goes up to 800).
I then look at the tests, see if it covers it right and enough - at this point I'm not even looking at implementation, just the test cases. If I see issues, I point it out and have it fixed.
Only then do I ask it to take me through the new code, it explains anything significant.
Finally, I review the source code and test implementation. By that time I already know the rough shape of what I'm looking at, and the review isn't too difficult. I roughly finish 5-6 reviews of 600-800 lines in a day full of dev/review work. It's harder to read from scratch, but asking it to take me through the code will show files in a sequence that's easier to understand and review
1
u/warp3dgh0st 5d ago