Updated June 2026

Somewhere along the way, models got a visible “working out” phase: Claude’s extended thinking, OpenAI’s o-series reasoning, DeepSeek-R1’s deliberation, Gemini’s thinking mode. (The product names churn fast; the durable fact is that every major lab now ships one.) Worth understanding what it is, when it earns its cost, and what it doesn’t promise.

What it actually is

Chain-of-thought used to be a prompting trick (“think step by step”). Reasoning models bake it in: they’re trained, largely with reinforcement learning, to produce an extended reasoning phase before the answer, and to use it well: trying approaches, catching their own errors, backtracking. The thinking is just tokens (same mechanism as everything else, see How LLMs Actually Work), but trained to function as scratch work rather than final prose.

The economic part: thinking tokens are output tokens, billed at the expensive rate, even when the provider hides or summarizes them. A model that thinks for 5,000 tokens before a 200-token answer costs 26x the naive estimate. This is the largest invisible line item in token costs.

When thinking earns its cost

The dividing line is roughly: does the task have intermediate steps that can be wrong?

Worth it: multi-step math and logic, hard debugging, planning a refactor, problems where the first plausible answer is often incorrect and self-checking pays. On these, reasoning models are not slightly better; they solve problems the same model fails without thinking.

Not worth it: lookup, formatting, extraction, classification, summarizing a provided document. The model thinks dutifully about a task with nothing to think about, and you pay latency and money for ceremony. A pipeline doing simple extraction with a reasoning model at full deliberation is the most common cost mistake of the current era.

Most providers now expose a dial rather than a switch: thinking budgets, effort levels, or adaptive modes where the model decides per request how much to deliberate. The practical pattern is per-route settings: high effort on the hard paths, minimal on the mechanical ones, decided by eval rather than vibes.

What the thinking text doesn’t promise

Two caveats that matter more as these models spread:

  1. The visible thinking is not a faithful window into the computation. What you see is often summarized, and even raw chains of thought are a generated artifact, not a log file. Treat it as useful signal for debugging prompts (you can watch it misunderstand your instructions, which is gold), not as ground truth about why the model answered as it did.
  2. Don’t build application logic on thinking output. Parse the answer, not the deliberation. Providers change thinking visibility and format without notice, and the thinking can explore wrong branches that the answer correctly discards.

And the perennial one: thinking raises the floor, it doesn’t install a guarantee. A model can reason its way to a wrong answer with great thoroughness. The verification habits don’t retire; they just trigger less often.