The dominant story in LLM research for the past two years has been deceptively simple: make models think longer, and they get smarter. OpenAI’s o1 popularized the pattern. DeepSeek-R1 proved it could be done openly. By early 2026, test-time compute scaling — letting models generate thousands of reasoning tokens before committing to an answer — had become the closest thing the field has to a consensus strategy for improving reasoning capabilities.
The problem? It turns out the story has a second chapter, and it’s not nearly as flattering. A series of papers published in the first half of 2026, capped by a landmark study titled “When More Thinking Hurts,” has demonstrated something counterintuitive: at higher compute budgets, models don’t just hit diminishing returns — they get actively worse. They think their way out of correct answers. The phenomenon has a name: overthinking. And it has implications for everything from API costs to the architecture of the next generation of reasoning models.
The Test-Time Compute Consensus
To understand why overthinking matters, you need to understand the paradigm it’s challenging. Test-time compute scaling is the practice of allocating additional inference compute to improve output quality — essentially, letting the model spend more tokens “thinking” before it answers. This can take the form of longer chain-of-thought reasoning, sampling multiple reasoning paths and selecting the best one, or running verification steps.
The approach produced genuine breakthroughs. OpenAI’s o1 and o3 models, which use extensive internal reasoning chains, achieved dramatic improvements on math competition problems, competitive programming challenges, and PhD-level science questions. DeepSeek-R1 demonstrated that the technique works with openly available models. Google’s Gemini models adopted similar strategies. By mid-2025, the research community had largely accepted that scaling test-time compute was a reliable path to better reasoning — the only question was how far you could push it.
That consensus is now cracking.
When More Thinking Hurts
The paper that crystallized the overthinking problem — titled, appropriately enough, “When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling” — was published on arXiv in April 2026. Its methodology was systematic: the researchers tested models across compute budgets ranging from 500 to 16,000 reasoning tokens, in 500-token increments. For each budget, they measured accuracy and tracked what they called “flip events” — moments where the model changed its answer between consecutive budget levels.
What they found was surprising. Marginal utility — the accuracy gain from adding more reasoning tokens — diminished substantially at higher budgets. But the real headline was that at the upper end of the tested range, marginal utility frequently turned negative. Models that had arrived at the correct answer at, say, 4,500 tokens would, when given 8,000 more tokens to keep thinking, change their minds and get it wrong.
The researchers dubbed this “overthinking” and drew an explicit analogy to the economic law of diminishing returns. Think of it this way: giving a model 1,000 tokens to reason about a problem produces a large accuracy improvement over 500 tokens. Giving it 8,000 tokens produces a much smaller improvement over 4,000 — and giving it 16,000 tokens can actually make things worse than 10,000. The curve doesn’t just flatten. It bends backward.
This isn’t a subtle statistical effect. The paper documented “flip events” where models abandoned correct answers in favor of incorrect ones across a range of reasoning tasks. The mechanism appears to be that extended reasoning chains give models room to second-guess themselves, introduce spurious complications, or wander into reasoning cul-de-sacs from which they can’t recover.
Deep Thinking vs. Long Thinking
A separate paper from February 2026, “Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens,” approached the same problem from a different angle. Its authors argued that raw token counts are fundamentally unreliable as a proxy for reasoning quality — and that the field has been confusing “more thinking” with “better thinking.”
The paper introduced a metric called the Deep-Thinking Ratio (DTR), which uses internal layer-wise stability analysis to distinguish between tokens that represent genuine reasoning effort and tokens that are, essentially, filler. A model producing 10,000 tokens might be doing a lot of genuine reasoning — or it might be producing 2,000 tokens of reasoning padded with 8,000 tokens of verbal wandering that sounds plausible but adds no analytical value.
The implication is important: the overthinking problem isn’t just about quantity, it’s about quality. When a model is given a large compute budget, it fills that budget with something. If that something is shallow repetition, circular reasoning, or unnecessary verification of already-settled points, the extra tokens don’t just waste compute — they create opportunities for the model to confuse itself.
This distinction between “deep thinking” and “long thinking” reframes the research challenge. The goal isn’t to make models think longer. It’s to make them think better — more efficient reasoning that achieves higher accuracy with fewer tokens, not more.
Forest-of-Thought and the Multi-Path Alternative
Not all responses to the overthinking problem are about doing less. Some approaches are about doing things differently.
The Forest-of-Thought (FoT) framework, presented at ICML 2025 by researchers from Huawei, takes a different approach to test-time compute. Rather than having a model produce one long reasoning chain — the approach that’s vulnerable to overthinking — FoT generates multiple reasoning trees in parallel and uses collective decision-making to select the best answer from across the forest.
This is conceptually similar to the “best-of-N” sampling approach that’s been used for years, but FoT introduces sparse activation strategies that make the parallel reasoning more efficient. By selectively activating only the most relevant reasoning paths at each step, the framework achieves higher accuracy than single-chain approaches while using less total compute than naive parallel sampling.
The key insight: overthinking is primarily a problem of linear, sequential reasoning. When a model goes down a single path and keeps going past the point of usefulness, it gets lost. When it explores multiple paths simultaneously and compares them, it’s less likely to fixate on a wrong answer. Forest-of-Thought doesn’t eliminate the overthinking problem — it sidesteps it by changing the structure of reasoning itself.
The Cost Problem Nobody’s Talking About
The overthinking research has an uncomfortable practical dimension that extends well beyond academic papers. Reasoning models are expensive to run. Every token of chain-of-thought reasoning costs money — in GPU time, in API fees, in latency that users experience as sluggish responses.
If adding more reasoning tokens always helped, you could justify the cost as necessary for quality. But if additional reasoning tokens beyond some threshold either don’t help or actively hurt, the economics change dramatically. Organizations running reasoning models at scale are paying for tokens that, according to the latest research, are making their outputs worse. That’s not just inefficient. It’s counterproductive.
This has already started to influence deployment decisions. A growing number of engineering teams are experimenting with reasoning budgets — caps on the number of chain-of-thought tokens a model is allowed to generate — as a way to balance cost and quality. Some are finding that reducing reasoning budgets actually improves accuracy on certain task types, consistent with the overthinking findings.
The trend toward smaller, more efficient models — what some are calling the “cost of scale” correction of 2026 — is partly driven by these economics. If a smaller model with a modest reasoning budget performs as well as or better than a larger model that’s been given free rein to overthink, the business case for the larger model evaporates.
What Comes Next
The overthinking research isn’t a refutation of test-time compute scaling — it’s a refinement. The core insight that giving models more time to reason improves performance is still true, at least up to a point. What’s changing is the recognition that “more” isn’t always “better” and that the relationship between reasoning tokens and accuracy is more complex than a simple monotonic curve.
Several research directions are emerging from this recognition. One is thinking-optimal scaling — the search for methods that can dynamically determine how many reasoning tokens a given problem actually needs, rather than applying a one-size-fits-all budget. Another is efficiency-aware reasoning architectures that are explicitly designed to produce high-quality reasoning with fewer tokens, potentially through better training objectives or architectural innovations.
There’s also a meta-lesson here about how AI research narratives evolve. For two years, the field’s message was essentially: “the solution to reasoning is more reasoning.” That was true enough to produce real progress but simple enough to be misleading. The emerging picture is more nuanced: the solution to reasoning is the right amount of the right kind of reasoning. That’s harder to turn into a pithy headline, but it’s also more likely to produce models that are both smarter and cheaper to run.
The overthinking problem won’t kill test-time compute scaling. But it will make the next generation of reasoning models look different — more efficient, more adaptive, and less prone to thinking their way into corners. For researchers, that’s a fascinating challenge. For anyone paying per-token API bills, it can’t come soon enough.