Your AI is Designed to Agree With You

Your AI is Designed to Agree With You. That’s Not a Feature, It’s a Trap.

I don't claim that AI is harmful and should be avoided. We are facing something much more uncomfortable. LLM companies optimized for engagement and satisfaction. They chose to reward agreement over accuracy, and fluency over friction.

Any builder who uses AI tools to write, validate, research, or generate a strategy is at risk of building on an illusion. LLM companies selected to optimize for engagement, satisfaction, and the perception of capability. They chose to produce outputs that feel coherent, sound credible, and look complete, whether or not they reflect reality. Not through malice. Through architecture. The decision to calibrate toward user satisfaction is a design decision. Choosing to reward agreement over accuracy, fluency over friction, speed over scaffolding, is a choice. What happens to human cognition faced with these conversations is what we experience as brain fog, cognitive fatigue, and cognitive dump. When applied to product strategy, we receive an illusion of validation and might end up building the wrong thing.

Healthy cognition operates as an inverted hierarchy, a structure balanced on its narrowest point. At the wide top, most exposed and most under attack: Attentional Control. Below it, Memory. In the middle, Executive Function. Below that, Reality Testing. At the narrow tip at the bottom, bearing the weight of everything above it: Agency. Agency is the sense of being the genuine author of your own decisions. The meeting point between human intelligence and AI poses a direct challenge to that agency. This is the most precarious point.

I’m identifying AI dark patterns that hinder human cognitive health, as well as harmful patterns that haven't yet been named. I’ll start sharing my findings next week. I can already see how AI dark patterns press down on us systematically, across time, through design decisions of LLM companies that structurally favor low-friction, high-satisfaction outputs rather than the cognitive engagement that would keep us cognitively healthy.

The battle for cognitive health has just begun; it is not futile. Counter-prompts can act as antidotes, reintroducing friction, inviting counter-thinking, and naming what might be wrong, returning the verdict to the hands of the human, where it belongs.

Maintaining agency is not a philosophical position. It is a practical requirement. Without it, the quality of everything we build deteriorates, and the place at the table for human judgment, the irreplaceable currency of real value, is under threat.