Thinking harder costs more, one token at a time

Why does thinking harder cost more? Thinking longer means more tokens, every token is a trip through the whole model, and every step of that trip is switches flipping. The bill counts the flips.

  1. 01The question
  2. 02Thinking is written out
  3. 03One token at a time
  4. 04One trip through the model
  5. 05What does the multiplying
  6. 06The switch
  7. 07Switches make logic
  8. 08Back up

01The question

Ask a model to think harder and it costs more. Pricing pages say so, and products offer "effort" settings that trade money for better answers. It sounds like a business decision: charge more for the premium answer. It isn't. The bill is counting work, and the work is physical.

This note follows that work down, from the bill for one reply to the smallest thing that does any of it, a switch. It builds on The GPU: many simple cores, which covers why models run on GPUs.

02Thinking is written out

A model writes in : a word, or part of a word, each. When it "thinks", it writes tokens before the answer starts: working-out, considering what you meant, checking itself. Some apps show it; others show a summary or a spinner. Either way the tokens are made.

Every thinking token is billed as output, the same as the answer, even the ones you never see. A higher effort setting lets the model write more of them. The answer at the end may be no longer, but the bill grows with everything written before it.

Fig 01
Low
1,500 tokens billed
Medium
4,500 tokens billed
High
16,500 tokens billed
Max
32,500 tokens billed

A printed line is 500 tokens. The answer is the same 500 at every effort; the thinking above it is billed as output too, at $20 a million.

One hard question at four efforts. The token counts are examples; the price is Opus 5.5's.

Thinking harder is writing more before answering, and every token written is billed.

03One token at a time

A model does one thing: given everything so far, it predicts the next token. Then it adds that token to the end and goes again. Your question, its instructions and every token it has written are its , and each new token is predicted from all of it.

That is what makes thinking work, though all it does is predict text. When the model writes "17 × 20 is 340", that line is now in the context, and the next prediction reads it. Writing the steps out gives each step the one before to stand on, the way working on paper beats working in your head. This is called , and on hard problems it measurably improves the answers.

Fig 02
0 trips through the model

Your question is in the context. Nothing has been written yet.

Grey: your questionThinking
A model working out 17 × 24, one token per press. Ink is the token just written; outlines are tokens already there.

Earlier tokens aren't worked out again for every new one. What was computed for them is kept, and the new token looks back at it. But each new token is still a full trip through the model of its own.

Reasoning is prediction used as a scratchpad: each token written becomes something the next one reads.

04One trip through the model

A token goes into the model as a long list of numbers and comes out as a probability for every possible next token. In between are layers, and each layer is mostly one kind of arithmetic: multiply the list by a large table of learned numbers, the model's weights, and add the results up. Each learned number is a .

So the work for one token is easy to estimate: about one multiply and one add for every parameter, two operations each, plus a little more the longer the context gets. A model with 400 billion parameters does about 800 billion operations for every token it writes, thinking included. Anthropic doesn't publish its models' sizes, so the sizes here are examples; the rule is from the research that measured it.

Fig 03
Parameters
4.0 × 10¹¹
Operations, one token
8.0 × 10¹¹
Operations, 4,500-token reply
3.6 × 10¹⁵
One token on one H100, flat out
0.81 ms

Every token pays the whole model again. Ten times the parameters is ten times the work, for every token, thinking included.

One token's trip through models of four sizes. Anthropic publishes no sizes, so these are examples.

Every token costs the whole model again: two operations for every parameter it has.

05What does the multiplying

Those operations run on a , a chip made of thousands of small arithmetic units that do the same sum on different numbers at once. An NVIDIA H100, a common one for running models, has 80 billion transistors and does about a thousand trillion of these operations a second.

Look inside one operation and it comes apart into smaller and smaller circuits. A multiply is many additions, an adder is a handful of (a common one-bit adder uses 28 transistors), and a gate is a few switches. An instruction lights one line builds an adder from gates.

06The switch

At the bottom is a switch that electricity flips: a current either gets through or it doesn't, depending on a signal. Computers have used three kinds, and they are not versions of one another. They are three different objects, each hired in turn for the same job.

  • The relay is a switch pulled by a magnet. A current in a coil pulls a metal arm across to close a circuit, with a click. It flips in about a thousandth of a second.
  • The vacuum tube has no moving parts. In a glass bulb with the air taken out, electrons flow from a heated wire to a plate, and a voltage on a mesh between them, the grid, stops the stream or lets it through. About a millionth of a second.
  • The transistor does it in solid silicon. A voltage on the gate opens a channel in the silicon underneath, and current flows. Under a billionth of a second, and small enough that a chip holds billions.
Fig 04
  1. 1Controlled byA current in a coil makes a magnet
  2. 2What switchesThe magnet pulls a metal arm down onto a contact, and it clicks shut
Years
1930s to 1940s
One flip
About a thousandth of a second
About the size of
A matchbox
Three switches hired for one job. Pick one to see what does the switching, and how fast.

A relay, a tube and a transistor are three different things hired for one job: a switch electricity can flip.

07Switches make logic

Wire two switches one after the other, and current gets through only if both are on: that is AND. Wire them side by side, and either will do: OR. Wire one so that when it's on it pulls the output away: NOT. Those three gates are enough to build any calculation, adders and multipliers included.

Fig 05

Relays, tubes or transistors: the table is the same. In a chip, NOT takes two transistors, NAND or NOR four, and AND or OR six.

Two switches wired three ways. Ink is current flowing; dots are none.

The truth table doesn't care what does the switching. A relay machine and a phone chip compute AND the same way. What changed over eighty years is how fast a switch flips, how little power it takes, and how many fit in one place.

08Back up

Now the right way up. Switches make gates, gates make adders, adders make a multiply, hundreds of billions of multiplies make one trip through the model, one trip makes one token, and thousands of tokens make a reply. Thinking harder adds tokens at the top, and every one of them is all the work below it again.

That is also why one model costs more than another. Opus 5.5 is billed at $20 a million output tokens and Haiku 5.5 at $0.50. A bigger model does more operations for every token, and a model allowed to think writes more tokens. Both multiply the same bill: a count of operations, each one switches flipping, run on electricity that ends up as heat.

The bill counts operations, and every operation is switches flipping. Thinking harder flips more of them.

Connected

Side threads

Questions that came up once the main idea began to hold, kept as they were asked. Each opens beside the paragraph it grew from.

Terms

Checked

What the chat had wrong or only half right, against sources. The note above already says the corrected thing.

  1. Modern transistors have a base, collector and emitter, and a small current switches them.That is the bipolar kind the first transistor computers used. Chips today are made of MOSFETs, with a gate, source and drain, switched by a voltage.Computer History Museum, 13 Sextillion & Counting
  2. Opus costs 15 times more than Haiku.Opus 5.5 is $20 a million output tokens and Haiku 5.5 is $0.50: forty times, by the prices of 25 October 2026.Anthropic, Pricing
  3. The model re-reads everything from scratch each time.What was computed for earlier tokens is kept, in a cache of keys and values. Each new token is one pass for that token, looking back at the cache.Pope et al., Efficiently Scaling Transformer Inference, 2022
  4. Claude hides its reasoning behind a spinner.Claude's apps show a summary of the thinking. The full thinking is still made, and billed as output.Anthropic, Building with extended thinking
  5. A vacuum tube is a thousand times faster than a relay.That compares whole machines, ENIAC against the Harvard Mark I. Switch for switch, a relay flips in milliseconds and a tube in microseconds.Britannica, ENIAC
  6. "Debugging" comes from insects jamming relays.Edison called faults "bugs" in the 1870s. The moth in the Harvard Mark II's log in 1947 was a joke on a word already in use.Wikipedia, Bug (engineering)
  7. The IBM 7090 replaced the 704's vacuum tubes, and ran the Moon landings.It was the transistorised 709. It ran the Mercury and Gemini flights; Apollo's were run on System/360 Model 75s.Wikipedia, IBM 7090; IBM, Apollo
  8. Without the 4004 there would be no Altair.The Altair ran on the Intel 8080, two generations on from the 4004.Wikipedia, Altair 8800
  9. The fridge-sized PDP-8 used early integrated circuits.The first PDP-8 (1965) was built from discrete transistors. The PDP-8/I (1968) used integrated circuits.Wikipedia, PDP-8
  10. Apple's M4 has 3 nm transistors.28 billion transistors is right. "3 nm" names the generation of the process; nothing on the chip measures 3 nanometres.Wikipedia, 3 nm process
  11. Opus is a physically larger model, trained longer.Anthropic publishes no sizes or training details. That Opus is the larger is a fair inference from its price and speed, not a published fact.Anthropic, Models overview
  12. MNIST and ImageNet were 2010s benchmarks; SWE-bench came in 2024.MNIST was published in 1998 and ImageNet in 2009. SWE-bench was published in October 2023.Jimenez et al., SWE-bench, 2023