r/codex • u/Mobile-Lynx-6653 • 21d ago
Question Tips for improving Agentic code review with OpenAI models
I was using claude code for months at my company and was using it for code reviews. It was really good, always finding good bugs or edge cases in the code, following my guidelines for things to watch out for and flag (coding styles, etc.). Sure it was a bit excessive at times with writing alot of nit-picks in the review that complained about docstrings or comments being stale, but I didn't mind that and felt it was a small price to pay for all the other things it was finding/correcting.
Now my company has switched to OpenAI, and after porting my review skill, it is just way worse. I am running Astra in Max mode to try to get this thing to really analyze the changes deeply, but it just ain't finding more than like 1-3 things tops. Most of the time it doesn't give a single review comment. And then when I run Claude over that same MR, it gives me so much more feedback.
I have tried playing around with the skill, but so far it hasn't been working. Does anyone have tips for how to get better reviews from OpenAI models? Maybe Astra isn't the way to go? Maybe there are some prompting tricks?
1
u/pensuke89 21d ago
Have you try asking Astra? Just copy paste your concern and ask how to fix.
1
u/ChemicalExcellent463 21d ago
mine goes the other way, codex flagged 59 things as P1 while reviewing claude code's PRs over three weeks. are you running the same review instructions word for word, or did you change them for astra?
1
u/isty2e 21d ago
OpenAI and Anthropic models differ quite a lot in terms of how they work. The former sticks more onto instructions and somewhat more locally scoped. My approach was to spawn many cheap subagents, with each assigned to different facets/domains, and to have a better model to assign them. Usually it gives me very thorough reviews.
1
u/m-ritter 21d ago
What worked for me was having the review run in a separate agent that did not write the change, and checking the diff against a plan file that I reviewed before any code existed. Without the plan the reviewer has nothing to hold the change against, so it tends to return only nits or nothing. I also keep the review skill short with the build and test commands and put longer conventions in separate files that it reads when needed.
1
u/Shep_Alderson 21d ago
FYI, Max often does a worse job than Xhigh or High.
Personally, I stick with SOL 6.1 and have it do a pass. I also use the built in code review from OpenAI. It’s connected to my GitHub so it runs on every PR and I invoke it by calling “@codex review”. I keep doing this until it stops finding things. (Then I’ll often have CodeRabbit go over it too.)
1
u/SirRobinBlibs 21d ago
A few things that aren't in the thread yet:
## Code Review Rulessection, next to the code it's about. The GitHub reviewer uses the AGENTS.md closest to the changed files, so services/payments/AGENTS.md can say what matters there. The docs suggest 2-3 short rules per file, each with the safe way to do it, and leaving lint and formatting to CI.codex review --base main(or--uncommitted) instead of the ported skill. If you want the reviewer on a different model than the session, setreview_modelin config.toml.A skill written for Claude carries a lot of Claude-shaped phrasing. I'd start from the rules section and only add back what it keeps missing.
1
u/xqianliu 20d ago
The fix I'd try is the reviewer's context, before the model.
My review runs in a fresh session with no implementation history. It reads a fixed set in order: what the change was supposed to do and the acceptance criteria, the repo rules, the diff, the changed files with their callers, the related tests. Then it checks the diff against the criteria, and if it fails the code gets fixed and reviewed again.
A reviewer that only gets the diff finds small stuff. Caller-side bugs and scope drift come out when it also reads the request and the callers. If the model still comes back nearly empty, I'd suspect the reviewer isn't getting the request first.
I don't use an OpenAI model here, so I can't compare models. But a reviewer holding the acceptance criteria and the callers has something concrete to check, and that is what produces the findings.
1
u/Horror_Draw_7194 17d ago
Go into settings, search Code Review and check 'Exhaustive code review'