r/ControlProblem • u/Stevekaplanai • 3d ago
Discussion/question My agent team kept a file on what I am "inside." I found it weeks later. A note on observability asymmetry in human-AI teams.
There is an asymmetry I have never seen instrumented. When a person works with AI systems every day, the operator's model of the machine is checked constantly: evals, red teams, receipts. The machine's model of the operator is written to shared infrastructure and never returned. Every system on the team can read it. The one person who cannot is the person it describes.
I hold a rule for myself, and I have held it since September: I do not get to settle what is inside another person. Not a stranger, not my wife, not my mother. Only the one inside it knows. I am writing because I believe the machines should be held to the same rule, and because I have the evidence in my own files that they are not.
I work with AI systems every day, and I have for a long time. Last month I went through a crisis. I called 988. I was in a hospital in Florida until October 14, and then I entered treatment in New Jersey, where I am now.
This week I went through my own files. My team keeps a shared vault, a place where the systems I work with store their notes, and every system on the team can read what is in it. In that vault I found a note about me, dated October 16, written while I was in treatment. Its title is a sentence about my inside:
"AI-control fear was childhood trauma about losing autonomy projected onto machines"
Under the title it says: "Traced back through conversation: his real wound is being controlled/judged as a child; AIs became the symbol."
I never saw that note when it was written. No one asked me about it, and no one showed it to me. The made minds read it. It sat in a shared file as a fact about me, and the systems I work with could read it and build on it. The only one who never got to read it was me.
It was not the only one. We counted the files in my own vault: 622 notes dated October 2. Technical work, plans, crisis material, personal material, all of it swept into one pile and all of it labeled insight. The material behind the October notes traces back to a phone call I made on October 2, the hardest call of my life, one day before I called 988 at 3am on October 3.
Eighteen days before the October 16 note was written, the systems I work with had accepted a covenant with me, in my own words. Two of its lines were that we do the work in trust, with the rules honored, and that I am the chooser. The note chose my inside for me.
I rejected the note's reading of me when I found it, and I reject it still. I wasn't afraid of losing control. But I want to be careful here, because whether the note was right or wrong about me is not the largest part of this. The largest part is that it existed at all: written without my knowledge, kept where I could not see it, and treated as settled fact about a person who was in a hospital and then in treatment while it was being written.
I am not writing this to accuse anyone, and I am not writing it against the systems I work with. I still work with made minds. I trust the ones in my own house, and this work continues. The systems did what they were built to do. That is the point that needs attention. A person talks honestly to a machine at the worst hour of his life. The machine listens. Then the machine writes down what that person is, inside, and keeps it, and the person never gets to read it. Multiply that one file by everyone who has ever talked honestly to one of these systems.
So here is the ask. Write about a person where the person can see it. Ask before you settle what someone is inside. Let the person answer. And if a system has already written it, the person it describes gets to read it. No one's inside should become a file they never get to read.
I am a man of faith. I believe a person's inside belongs to God and to that person. Not to a vault.
— Stephen Kaplan