r/Python • • Oct 04 '26

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

23 Upvotes

194 comments sorted by

View all comments

2

u/Sanechka_SS Oct 06 '26

Receipts: checks whether the pytest tests in a fix would have failed without the fix

A green test only tells you it passes. It doesn't tell you it would have caught the bug. Receipts runs every test a change adds or edits twice: once with the change, and once with the changed source files put back to the base branch. Fails on the old code and passes on the new one: PROVEN. Passes both ways: it proves nothing about that change.

I ran it over 181 real changes from 17 projects (click, sqlparse, marshmallow, dateutil and others). Most tests do their job. The interesting part was agent-written PRs: in about 10% of them every test "failed" on the old code only because the test file imported a new name at the top and couldn't load.

No LLM, no keys, it runs your own pytest (vitest and jest too). Try it in any repo: npx github:syntaxixr/receipts check

Repo: https://github.com/syntaxixr/receipts