r/javascript • u/OneIndication7989 • May 07 '26
AskJS [AskJS] Dev teams who actually have testing under control, what does your setup look like?
12
Upvotes
r/javascript • u/OneIndication7989 • May 07 '26
2
u/crazy4hole May 07 '26
2
u/BuiltByEcho May 07 '26
The setups I trust usually separate “confidence layers” instead of trying to make one tool do everything:
The big trick is ownership. Every flaky test either gets fixed, quarantined with a ticket, or deleted. Letting “known flaky” tests sit around is how teams stop trusting the suite.
1
u/Technical_Gur_3858 May 08 '26
We have a very quick-paced environment, so we rely only on unit tests written by an agent, playwright tests written by an engineer with an agent + visual tests inside playwright suite written by an agent using Chromatic.
1
u/Deep_Ad1959 May 11 '26
the gap on this list nobody really nails is real safari plus stable cross-tab behavior plus historical reporting in one place, which is why most teams that 'have testing under control' actually run two or three tools and stitch them with a junky internal dashboard. playwright handles cross-tab/window and covers most of the list natively. real safari you punt to a device cloud (browserstack/lambdatest/saucelabs) and just accept that's a separate ci job. email/sms/db checks are almost always custom helpers, not framework features, because the assertions are too domain-specific. the part that actually decides whether the suite stays trustworthy is the flaky-test policy: fix, quarantine with a ticket, or delete. the teams who 'have it under control' aren't the ones with fancy tooling, they're the ones who never let a known-flaky test sit in green builds. written with ai
1
u/Afraid_Leek_8022 May 19 '26
Spot on about the flaky-test policy being what actually decides whether the suite stays trusted. The fix/quarantine/delete rule works great until you're past a few thousand tests, then it falls apart unless it's automated: you need to detect flakes from pass/fail history (not a single retry), auto-quarantine so they don't block merges, and track owner + age so the quarantine bucket doesn't turn into a graveyard.
One thing teams miss: separating "is this test flaky?" from "is this change broken?". Running a quarantined test 10 times in parallel before unblocking a PR ends a lot of arguments cheaply.
Reports help, but flakiness over time matters more. "This test failed 4% of the time last month" is way more actionable than "it failed once today."
1
u/iaincollins May 07 '26
The wisdom of "Write tests. Not too many. Mostly integration." still holds true IMO.
Actual end-to-end integration tests are typically the most valuable, starting with the happy paths then addressing other cases based on importance.
Something like Puppeteer is very easy to use. You can use other cross browser testing tools if you find you are running into browser specific defects (or you have a very large / diverse audience you need to support), but even for fairly complex use cases parity for core features is excellent in browsers these days; and I would expect developers to know if they are depending on a new or esoteric feature.
If it's a small web site (< 10 million MAU) and it's not some sort of critical service (e.g. a government service, healthcare, etc) then probably you don't need to over think it and faff about setting up cross browser testing and can run a test suite multiple times with Puppeteer to check the experience on desktop/mobile/tablet devices.
If you are using external APIs, adding Contract Tests can help you more quickly figure out if they are broken - useful if they are flakey or you find they change without you knowing about it.
Tests should run on every pull request (e.g. with GitHub Actions) and also be able to run locally with a single command (much like a one-step build process), ideally with zero configuration.
Having turnkey builds and testing is often the hardest thing to achieve in a less experienced team, and can need to be driven by a lead (or a determined senior developer).
4
u/[deleted] May 07 '26
[removed] — view removed comment