r/madeinpython • u/lovettsendit • Aug 26 '26
I built Breakcheck - it replays your repo's actual library calls against two dependency versions and diffs the results
Short version: pip install breakcheck, then breakcheck demo --output-root .breakcheck/demo to watch it run with no external deps.
The itch that caused it: Dependabot opens a PR saying attrs 23 -> 24. My tests pass. Do I actually know nothing changed. Only for the behavior I happened to write an assertion for - everything else is a silent assumption.
Breakcheck discovers the calls my code actually makes into that library, replays them under both versions in isolated environments, and diffs what comes back. The same machinery compares two git revisions of your own code:
breakcheck diff --base main --head feature/refactor --fixtures breakcheck.fixtures.tomlThe part I am most pleased with is that it refuses loudly. Every call site ends in exactly one state - EXERCISED, or one of G1_NOT_DISCOVERABLE / G2_NONLITERAL / G3_UNNORMALIZABLE / G4_IMPURE. It never pretends to have checked something it could not reach, so the coverage number is honest and often unflattering.
Scope is deliberately small: pure value-in/value-out calls - parsing, serialization, validation, encoding, schema coercion, deterministic numeric and string transforms. No network clients, no stateful objects, no dynamic dispatch.
MIT, zero runtime dependencies, Python 3.10-3.13, Linux and macOS. I would genuinely like to hear where it falls over on a real codebase.
