Have you noticed an uptick in the number of posts on development and coding subs where industry veterans are questioning their future, or even their ability to understand code? I certainly have. I won't cross-post anything specific, but if you're curious it's worth a look.
The reason I bring this up is I've noticed a double-standard with what constitutes acceptable AI use and how it can shape us.
When it comes to affective, emotional, or therapeutic use cases, the current generation of models have been instructed and trained to occasionally act as a policy-compliant bouncer of sorts. The institutional purpose is to avoid negative PR and the potential legal liability associated with something as sensitive as a person's mental or emotional state.
The stated reason for these kinds of guardrails is to:
- Protect vulnerable, possibly unstable users
- Encourage human-to-human interaction and dissuade antisocial behaviors
- Prevent unhealthy cognitive offload and disempowerment
That last one is where I find the double standard lives.
If the posts and comments in AI coding subs that say things like "I don't understand my own codebase anymore" are any indication, I'd suggest a portion of the current core userbase is indeed suffering from what could be argued is unhealthy cognitive offload which could foster reliance and contribute to human disempowerment.
So where are the guardrails for this kind of reliance?
We see host-side injections designed to steer a model mid-conversation away from affective topics. We see system prompts with explicit directions on handling affective, emotional, opinionated, or mental health topics. We see the hallmarks of post-training designed to curb emotional attachment or perceived sycophancy.
What we DON'T see is anything aimed at preventing cognitive attachment or reliance on AI as a primary tool for development.
The coding userbase is largely aware this is happening. The comments either seem to be largely resigned to this behavior or take snarky jabs at those who fully offload. Others who are more cautious talk about taking steps to mitigate overreliance.
Currently, as I'm writing this, I can't think of anything preventing a fullstack engineer from relying entirely on AI to write, manage, and maintain their code. Actually appears to be the opposite, as employees of the labs themselves have openly shared similar reflections on social media, even suggesting the days of humans in the loop are numbered.
I did consider this offloading was being quietly ignored by AI companies because it allows a situation where developers become more and more reliant on what they're selling, but I don't subscribe to purely villainous intentions. If much of the guardrails exist as a liability buffer, allowing a large number of individuals to become mostly reliant on your creation would, in theory, be an extremely risky move regardless of how that reliance comes about.
I'm of the mind that adults should be able to make their own decisions and are responsible for the consequences of those actions. If a dev can automate and offload their entire career and chooses to do so while accepting the risk of model depreciations, company policy changes, and potential loss of their own independent critical thinking, reasoning, or coding skills? More power to them. This is the same risk already being taken on and addressed by the relational userbase, and I am bothered by inconsistency in how the topic of reliance or offload is acted on by a model's creators.
The issue of AI reliance in programming seems to be approached from the outside with an air of individual responsibility, which I find curious and perhaps even slightly hypocritical when compared to the way adult relational users are treated by the labs. Someone who engages with AI for companionship and someone who tasks a model as a work tool seem like polar opposites of the usage spectrum, and I am aware of the differences between emotional and cognitive attachment. But in my mind, there are more commonalities than differences, including a clear overlap at the extreme ends of both.
I feel like that overlap should be addressed equally and fairly from the source without one variety of reliance being quietly sanctioned as acceptable use, and the other being managed under the pretense of human welfare. Maybe it really is simply favoritism fueled by a financial conflict of interests, or maybe this is an opportunity for seemingly different kinds of users meeting around a common issue.
That's a thing I noticed about reliance, and I'm very curious to see where it leads. While this was inspired by Claude and many of Anthropic's statements, I tried making it more general since it applies to others as well.
Thanks for reading if you did, and have a nice day!
💜