Chain of Thought

The mechanism is simple and worth understanding literally: a model has a fixed amount of computation per token, so a problem that needs several logical steps cannot be solved inside a single token. Generating the steps spreads the work across many forward passes and puts each partial result into the context where later tokens can attend to it. This is why the technique helps most on arithmetic, multi-constraint reasoning and planning, and barely at all on recall or single-step classification.

In application design the question becomes what to do with the reasoning. Showing raw intermediate steps to end users is usually a mistake — it is verbose, occasionally wrong in ways that undermine a correct answer, and invites debate about the process rather than the result. The common pattern is to let the model reason and then emit a clearly delimited final answer that the application extracts, keeping the reasoning for logs and evaluation. Models with an explicit extended-thinking mode formalise this separation at the API level.

The misconception is that the stated reasoning explains the answer. It does not reliably. The written chain is generated text, not a transcript of an internal process, and a model can reach the right answer through a chain containing a wrong step, or narrate a convincing chain that leads to a wrong answer. Treat it as a computational aid that often improves results, not as an audit trail you can trust.