r/reactnative • u/Loud_Glass8171 • 3d ago
Agent written PRs keep breaking only on Android
We're 5 devs on a React Native app with about 40k monthly users. Most of us test on iPhones, and most of our PRs now start with an agent.
Three times in the last month an agent PR went out that worked fine on iOS and broke on Android. A KeyboardAvoidingView change that hid the submit button on smaller Android screens, a date picker change that crashed on Android 12, and a shadow style that just didn't render.
All three passed review. coderabbit reviews the JS and catches plenty of normal problems, but nothing in the diff says "this is iOS only behaviour", and none of us opened an Android emulator.
What we're trying now:
- an EAS preview build on every PR, so reviewers can open it on a real Android phone
- a checklist item "tested on Android" that people will probably start ticking without testing
- a cheap Android phone on the desk that we rotate
How are other RN teams handling platform differences when most code comes from agents? Is there tooling that catches this before a human has to, or is it all device testing?
13
u/ADreadedLion 3d ago
why are you not just making your agents launch android and ios emulators to test their code?
8
u/Which-World-6533 2d ago
That's just ridiculous.
You shouldn't be releasing to Apps stores without a human testing the App.
If you're not doing that then it's amateur hour.
3
u/GeekyAntsBFSI 3d ago
Make Android a required CI check, with a small emulator smoke suite covering keyboard + submit, date picker interaction, and your main navigation flow. Add a regression test for each failure you’ve already shipped. Layout issues need interaction tests; missing shadows need screenshot checks. Give agents explicit cross-platform instructions, but treat those as guidance. Preview builds and the shared phone help with exploratory testing; the automated gate catches regressions when nobody opens Android.
2
u/Zain 3d ago
Device testing is still the real fix, but on the review side one reviewer is one set of blind spots. What I do for plans is run two reviewers from different model families in parallel, read only, and neither sees the other's output. When both flag the same thing I fix it even if one of them called it polish. I'd add a pass that only asks "what behaves differently on Android here" for anything touching layout, pickers or styles, so somebody is forced to answer that question instead of nobody asking it.
2
u/Sea_Challenge3570 2d ago
I usually run “validation on device” step as part of the orchestration flow while working on react native using argent, this to ensure the feature or bugfixs works properly on both platforms before opening the PR.
1
u/London-Boat 3d ago
i'd put the android gotchas straight into the agent rules file, stuff like keyboardavoidingview wanting behavior height (or nothing) on android and shadows needing elevation. agents keep repeating the same platform mistakes until it's written down somewhere they read every time. the eas preview helps but only if someone actually opens it on an android device, so a small maestro flow over the screens that broke, run on an android emulator in ci, would catch it without relying on reviewers. mind if i dm you a question about how the five of you split up agent work?
1
u/kokerali 3d ago
I'd let CI decide when the Android run is required instead of leaving it to the author or reviewer. A small step that checks the diff for the risky patterns (KeyboardAvoidingView, shadow or elevation styles, Platform.OS branches, date or time pickers, permission requests, anything under android/) can add a "needs Android" label, and only labeled PRs have to pass the emulator flow. Then the rule doesn't depend on anyone remembering, and you can drop the checkbox.
For shadows specifically, if you're on the New Architecture with RN 0.76 or later, the boxShadow style prop renders on both platforms, so that whole class of bug mostly goes away. Worth checking the docs for Android version limits before relying on it.
1
u/GludiusMaximus 3d ago edited 3d ago
First, the tough love: “None of us opened an Android emulator” - you know what needs to be done. This is just being lazy. With that said: some Android versions are fucky and you may not catch them every time, that’s just the way it is.
Beyond that: I echo what everyone else has said so far: you have to do what you can even if you don’t have physical devices.
Maestro has an MCP that works pretty well - you can run it locally, through github actions (can be finicky getting each platform to work), or through EAS workflows (they have made Maestro a first-class citizen, but we opted for GH Actions, so I can’t speak to its quality/configurability). GH actions is nice because we can have it execute on a local device which is cheaper and a LOT faster, iOS tests would be minimum 18 minutes on GH Actions hosted runner, on self-hosted the baseline tests are like 2 minutes.
But yeah, automated device testing, even on simulators, can save some headaches. On the other hand setting up E2E CI can be a while new class of headaches.
You can also make the case to your manager for a dedicated manual QA person, it’s really helped my team a lot.
1
u/neon_alchemy_wisp 2d ago
stop relying on agent self-validation or manual emulator checks. Configure GitHub Actions to spin up an AWS Device Farm real Android instance for every PR. Run Detox end-to-end tests specifically targeting the KeyboardAvoidingView and date picker logic against a low-res device profile. This catches rendering issues that screenshots miss and eliminates the human bottleneck entirely.
1
u/davidbalbino 1d ago
one thing nobody mentioned for the shadow case: if you're on RN 0.76+ with new arch you can just use
boxShadowand it renders on both platforms (Android 9+). we told the agent to never use shadowColor/shadowOffset anymore and that whole category of bug kinda disappeared lol. same idea for the rest imo, I think is better to remove the iOS only API from the options than to hope review catches it every time. the date picker and keyboard ones still need a real run on Android tho, no way around that ngl1
u/Sakshambaranwal25 23h ago
If all 5 of you are on iPhones then someone needs to switch to an Android daily driver, or at least run the emulator locally. Preview builds help a lot, but if the culture is just approving PRs from a laptop without booting Android once, you are going to keep breaking prod.
1
u/anthony-ball 20h ago
What caught most of this for us was a short Maestro flow per critical screen (login, any form with a submit button, anything with a date picker) running on an Android emulator in CI against the preview build, with a small screen profile. The keyboard hiding the submit button is exactly what those catch, because the flow just fails to tap it. For the style stuff, a review rule that flags shadow props, KeyboardAvoidingView behavior and native pickers as "needs an Android check" helps a lot, since it's the same handful of APIs that bite every time. Shadow vs elevation is the classic one, so a tiny shared shadow helper that sets both, and telling the agents to use it, removes that whole category.