Your AI Agent Will Never Question the Premise

Third in a short series on three lenses for thinking under pressure: tension, connection, and reduction.

Open a fresh chat. Ask a capable agent to design your product's pricing. What comes back is competent, confident, and reasonable: three tiers, a per-seat model, an annual discount, and an enterprise "contact us." It looks like the pricing page of every company in your category, because that is exactly what it is.

That is not the agent being lazy. It is the agent working precisely as built. A model predicts the most probable next word, then the next, all the way to the end of the answer. Stretch that across a whole pricing strategy, and "most probable" becomes "most common," which becomes the center of everything the model has ever seen. The technical name for the pull is regression to the mean. One 2025 study put hard numbers on it: across 2,200 essays, each new human-written one added roughly two to eight times as much fresh variety to the collective pool as each new GPT-4 essay did, and the model's drift toward the middle persisted even when the researchers threw every clever prompt at it.

The fluency is real. So is the gravity toward the average. And the average turns out is about to become the most worthless thing in business.

There is a discipline for refusing the average answer, and the cleanest demonstration of it is also the most over-told, so I'll be quick.

In the early 2000s, a rocket to orbit cost around $65M, and the industry explanation was, in effect, that rockets have always cost that much. Reasoning by analogy. SpaceX asked a different question: what is a rocket actually made of, and what do those metals and that fuel cost on the open commodity market? The answer came back at roughly 2% of the sticker price. The other 98% was not physics. It was an accumulated convention nobody in the industry could still see, because it had been there the whole time. Musk's phrase for the move was to "boil things down to the most fundamental truths" and "reason up from there," instead of reasoning from what everyone already does.

I call this lens reduction. It's also known as first principles, and Aristotle called it archai. Not reductionism, the bad habit of explaining everything away. Not the cost-cutting kind that haunts every budget review either, which is the meaning the word unhelpfully suggests in an operating context. Reduction means getting down to bedrock truth and rebuilding from it. Take the thing apart until you hit what is actually true, the hard floor of physics or unit economics or the real job the customer is hiring you to do, then reason up and ignore the conventions stacked on top.

Notice what the move is not. It is not optimization. An agent optimizes beautifully; point it at your pricing, and it will polish the existing shape until it gleams. Reduction asks whether that shape should exist at all. Answering that means standing outside the cloud of every prior answer, which is the one place a next-word predictor structurally cannot go. The model lives inside the cloud. That is its whole job.

The lazy reading is that AI makes thinking cheap, so thinking matters less. The economics run the opposite way.

When the average competent answer drops to near-zero cost and shows up in four seconds, it stops being worth anything, precisely because everyone now has it. Your competitor has it. Your competitor's competitor has it. You are all prompting the same handful of models with the same unexamined questions and receiving the same average back. What holds value is the answer that sits off the average, and reduction is how a human gets there.

Here is the part I did not see coming when I started drafting this piece. The same decomposition that helps a human find a better answer is also what makes the AI agent more reliable. The 2025 research on agentic systems is consistent on this: break a task into its fundamental sub-problems, and accuracy goes up while hallucination goes down, because each piece is now small enough to handle cleanly. One method that decomposes a task by its formal complexity lets an agent perform measurably better on hard combinatorial and database-querying benchmarks.

Sit with that for a second. The human move that produces a non-obvious answer and the engineering move that makes an agent trustworthy are the same move, performed at two altitudes. Reduce the problem to fundamentals, hand the well-framed pieces to capable agents, and you get genuinely superhuman output. Skip the reduction, hand over the problem exactly as you inherited it, and you get superhuman speed at scaling the wrong thing. The framing is the whole game, and it's the human's job.

Now the part that costs real money this year.

An agent will execute a flawed premise with total confidence, because nothing inside it is built to doubt the frame you handed it. It cannot tell the difference between a good question and a bad one. It can only answer fluently. So a bad question yields a fluent, well-structured, completely wrong answer, delivered fast and with no tell.

The data is already uncomfortable. In one large 2025 developer survey, the top frustration with AI coding tools, named by two-thirds of respondents, was output that is "almost right, but not quite." Nearly half said debugging that AI-generated code takes them longer than writing it themselves would have. (Yes, the models have improved significantly since, yet the underlying architecture remains.) That failure has an old name in aviation: automation bias. People follow a confident wrong instruction from a machine at a meaningfully higher rate than they make the same mistake on their own, and the effect is worse for the least experienced, the very people least able to tell a plausible answer from a true one.

The root of the verification problem is singular. You cannot catch a confident wrong answer unless you can reason from fundamentals yourself. Reduction is the only way to check polished output against bedrock, rather than against whether it "looks right," which is exactly the test a fluent wrong answer is built to pass. The leader who can still take a problem apart is the one who can govern an agent. The one who has let that muscle go ships the plausible-wrong answer at machine speed and finds out a quarter later.

Reduction has a cost, and the discipline includes knowing when not to pay it. Taking a problem to fundamentals is slow. For the reversible, low-stakes, genuinely solved decision, reasoning by analogy is correct precisely because it is fast, and handing it to an agent unexamined is exactly right. Reserve reduction for the high-stakes call, the stuck problem, and the contrarian bet, where convention is most likely to be hiding something.

Two cautions, because first-principles enthusiasm reliably produces both. First, some conventions are load-bearing. They encode a hard-won reason that is no longer visible, and tearing one down because you cannot see its purpose is a self-inflicted wound. Before you discard an assumption, find out why it exists. If nobody can say, you have found a real opportunity. If somebody can, you may have found a fence worth leaving standing.

Second, reduction can be wrong about the bedrock itself. Reason confidently up from fundamentals that are incomplete, and you get an answer that is internally flawless and externally false, a map drawn so cleanly you stop checking it against the ground. The fundamentals you reduced to are still a model of the thing, not the thing. The defense is the same one that catches a confident agent: test the rebuilt answer against reality before you trust it.

Pick one decision you are about to hand an agent this week. Your pricing, a market-entry plan, a hiring profile, a workflow you mean to automate. Before you prompt anything, write down the single biggest assumption baked into how you framed it. Then ask one question: is this a hard constraint, the floor of physics, unit economics, the real job the customer hires you for, or is it a convention everyone in your category has simply stopped questioning?

If it is a convention, you have found the work the agent cannot do for you, and the work most worth doing. Reframe from the fundamental truth up, then hand the agent the pieces. You will get a different answer than your competitors, all of whom are about to ask the same model the same unexamined question and get the same average back.

The machine is extraordinary at giving you the best version of the existing answer. It just cannot tell you when the existing answer is wrong. That part is still yours.

Elon Musk, quoted in James Clear, "First Principles: Elon Musk on the Power of Thinking for Yourself," jamesclear.com. https://jamesclear.com/first-principles Accessed 2026-06-11.

Catherine Clifford, "Billionaire Elon Musk says this is 'a powerful, powerful way of thinking'," CNBC, 2020-02-28. https://www.cnbc.com/2020/02/28/billionaire-elon-musk-this-is-a-powerful-way-of-thinking-but-hard-to-do-how-it-works.html Accessed 2026-06-11.

Shane Parrish and Rhiannon Beaubien, The Great Mental Models, Volume 1: General Thinking Concepts, Farnam Street Media, 2019. ISBN 9781999449001.

Kibum Moon, Adam E. Green, and Kostadin Kushlev, "Homogenizing effect of large language models (LLMs) on creative diversity: An empirical comparison of human and ChatGPT writing," Computers in Human Behavior: Artificial Humans, vol. 6, 2025, 100207. https://doi.org/10.1016/j.chbah.2025.100207

"Galton's Law of Mediocrity: Why Large Language Models Regress to the Mean and Fail at Creativity in Advertising," arXiv:2509.25767, 2025. https://arxiv.org/abs/2509.25767 Accessed 2026-06-11.

"An Approach for Systematic Decomposition of Complex LLM Tasks," arXiv:2510.07772, 2025. https://arxiv.org/abs/2510.07772 Accessed 2026-06-11.

Burak Gozluklu, "How task decomposition and smaller LLMs can make AI more affordable," Amazon Science, 2024-09-19. https://www.amazon.science/blog/how-task-decomposition-and-smaller-llms-can-make-ai-more-affordable Accessed 2026-06-11.

Stack Overflow, "2025 Developer Survey: AI," via ShiftMag, "84% of developers use AI, yet most don't trust it," 2025. https://shiftmag.dev/stack-overflow-survey-2025-ai-5653/ Accessed 2026-06-11.

Addy Osmani, "The reality of AI-assisted software engineering productivity," 2025. https://addyo.substack.com/p/the-reality-of-ai-assisted-software Accessed 2026-06-11.