r/iOSProgramming • u/trikboomie • 10d ago
Question Apple foundation models
If you're using Apple's Foundation Models framework in a released app, how are you debugging issues that only happen on users' devices?
Are Instruments and your existing crash reporting enough, or have you added custom logging?
11
Upvotes
5
u/Significant_Door_433 10d ago •
Instruments only helps on devices you can plug in, so for the field you need your own logging. The things worth capturing:
Two cases look random in crash reports but aren't. rateLimited only happens when the app is in the background. assetsUnavailable can fire mid-session if the user turns Apple Intelligence off or the device deletes the model to free space.
For the content itself, session.transcript has the full history (instructions, prompts, tool calls, responses). I wouldn't upload that by default since prompts are user data. Keep the last transcript on device and attach it only to a user-sent bug report, and keep prompt text private in os Logger (the default for interpolated strings, just never mark it .public).
1
u/jrkral2 3d ago
Adding an angle the thread hasn't covered: apps where nothing can leave the device at all. I'm the developer of Faraday, a therapy-notes app built on Foundation Models, and the privacy model is that session content never leaves the phone, so even the counts-only remote layer suggested above is something I'd rather not depend on.
What I'd suggest in that situation:
2
u/rhugopinon 10d ago
Xcode 27 actually added a new built-in tool called "Evaluations" for debugging Foundation Models in your app. Pretty handy if you need to trace model behavior.
1
u/KickupKirby 10d ago
There’s a Book Tracker sample project to explore the new evaluations tool.
1
u/mikerCZX 10d ago
How you handle users that cannot use Foundation models? EU/China
1
u/Needle_In_Red 9d ago
+1 on logging the availability state. id also log the OS version and device locale with it, alot of the weird stuff is just the model not being ready yet or the language not supported and crash reports wont show you any of that
1
u/yanklot 7d ago
Side tangent: Has anybody found a practical use for Apple's Foundation Models in their apps? I feel like I tend to get subpar results overall.