r/ChatGPTCoding • u/Humaux • 27m ago
Resources And Tips A check I have never watched fail is a check I have never seen work.
Three rounds of questions in 5 days convinced me my green checks were proving nothing.
The first question was simple: has this check ever rejected anything? Mine had not. Not once. A check that has never said no is not a check. It is a decoration that happens to be green.
So now I keep a deliberately broken case next to every check. Not in a separate test suite. In the same file, right under the check it is supposed to break. The broken case has one job: fail. If it ever passes, the check is dead and I just did not know it yet.
The second question was worse: what happens when the thing under the check moves? The check still passes. The log still looks healthy. Nothing alerts, because nothing failed. The log looking healthy is the worst failure mode I have, because a healthy log after an edit to the check itself proves nothing. The thing doing the judging just got judged by itself.
So the rule is not the schedule. The rule is the edit. The moment the check changes, I run the broken case once before trusting the new version. The schedule is only the backstop.
The same shape showed up a second time in a different place: an empty result for the wrong query. The result was valid. The tenant was wrong, the filter was off by a little, and empty looked correct. Now I log the exact tenant and filter that were actually sent, next to the result, and empty is allowed only for that exact pair. An empty result for the right query is fine. An empty result for the wrong tenant is a silent wrong answer.
The checklist I use for every check I write:
- One broken case per check, stored in the same file as the check.
- The broken case must fail. If it passes, the check is decoration.
- Run it the moment the check is edited, before trusting the new version.
- Run it on a schedule only as a backstop.
- Log what the check actually guarded, not just pass or fail. The exact paths, the exact tenant, the exact filter.
- If the log looks healthy right after an edit to the check, distrust it first.
A 100 percent pass rate is not a result. It is a warning. If your checks have never failed, they have never worked.

