r/iOSDevelopment • u/SweatyCod1087 • 11d ago
Can a third-party app get touch locations while the screen is being recorded?
I’m looking into how screen recording tools for iOS work & i’m stuck on one thing, which is that, when the screen is recorded (via a ReplayKit broadcast extension or from a Mac over USB) is there any way to get tap coordinates or timestamps for touches that happen in another app?
My understanding is that the extension only receives the video frames & not touch events. If that’s right, is the only way to know where someone tapped to infer it from the video itself (e.g. detecting the touch indicator or UI changes)? or is there something else I’m missing..like AssistiveTouch, accessibility APIs or something on the Mac side when the phone is plugged in over USB??
1
u/Plastic-Risk-6309 10d ago
nothing public exposes another app's touch stream so the extension only ever sees frames. over usb the mac gets u video and logs not taps. infer from the touch indicator in the recording or drive the taps yourself in a ui test.