The Journey Behind Every AI Answer

[ 01 · Your Phone: From Words to Tokens ]

The first thing that happens to your question is that it stops being a sentence.

Language models do not read letters or words. They read tokens — chunks of text roughly four characters long. "Unbelievable" might arrive as three of them. A comma is usually one. Your phone hands the model a list of numbers, each one an index into a fixed vocabulary, and the model never sees your prose at all.

Before that list leaves the device it is encrypted, then chopped into packets: small bundles of data, each stamped with a destination address, each free to take its own route. A typed question is a tiny payload — smaller than a low-resolution photo. It goes out over Wi-Fi or cellular to a router, and from there into the public internet.

Your phone, in other words, does almost none of the work. It is a keyboard, a screen, and an envelope. The thinking happens somewhere else, and the somewhere else is usually several hundred or several thousand miles away.

[ 02 · The Network: Two-Thirds the Speed of Light ]

Light moves slower in glass than in vacuum — about 200,000 kilometres per second, roughly two-thirds of its top speed. That number sets a hard floor under every online interaction you will ever have. Nothing you do on the internet can beat it.

Most of the distance is covered by fiber optic cable, and the most important stretches are underwater. There are more than 800,000 miles of submarine cable on the seafloor — more than three times the distance from the Earth to the Moon — and they carry roughly 99% of all intercontinental data. Not satellites. Cables the diameter of a garden hose, lying in the dark, occasionally bitten by fishing gear.

In practice a packet takes something like 10 to 50 milliseconds to reach a data center, depending on how far it has to go. A blink takes about 100. Low-Earth-orbit satellite constellations offer an alternate route for places fiber does not reach, bouncing your prompt up and back down instead of around.

[ 03 · The Power Layer: The New Bottleneck ]

For most of the AI boom, the scarce thing was chips. That is no longer true. The scarce thing is electricity, and the equipment needed to move it.

The four largest cloud companies have guided to more than $600 billion of capital spending in 2026, up from around $388 billion in 2025 — and closer to $660–690 billion once Oracle is counted. That works out to roughly $1.7 billion per day, every day, mostly poured into buildings, transformers, cabling and silicon.

The result is a new category of construction. In 2026, five separate AI campuses cross one gigawatt of power capacity: Anthropic and Amazon's site in New Carlisle, Indiana; xAI's Colossus 2 in Memphis; Microsoft's campus in Fayetteville; Meta's Prometheus; and OpenAI's Stargate site in Abilene, Texas, targeting 1.2 GW and more than 450,000 GPUs.

A gigawatt is roughly the output of a full-size nuclear reactor. Five reactors' worth of demand, appearing at five specific addresses, on a grid that was not planned for them.

You can order more chips. You cannot order a substation and have it next quarter. Interconnection queues — the waiting list to plug a large load into the grid — run four to seven years in the markets that matter, and roughly half of planned US data centers face delays or cancellation. Transformers and switchgear, unglamorous steel and copper, have become gating items. The frontier of AI is now partly a construction and permitting problem.

[ 04 · The Data Center ]

Your packet arrives at a building the size of several football fields, usually with its own substation out front and its own water infrastructure out back.

Inside, the layout is monotonous on purpose: rows of racks, each rack a steel cabinet of computers, each aisle carefully separated into hot and cold. The interesting engineering is thermal. Modern AI chips consume so much power in so little space that blowing air past them no longer works. The heat has to be carried away by liquid — cold plates bolted directly onto the chips, with coolant circulating through them and out to the building's cooling loop.

From there the heat usually goes into evaporative cooling towers, which dump it into the atmosphere by evaporating water. This is why AI data centers show up in local water debates as often as local power debates: a large campus can consume serious volumes of water, particularly in hot, dry regions where land and power happen to be cheap.

Everything else in the building exists to keep the computers from ever stopping: backup generators, banks of batteries, redundant power paths, redundant network paths. A gigawatt campus is, functionally, a small industrial city whose only product is heat and tokens.

[ 05 · The GPU Rack: 72 Chips Pretending to Be One Brain ]

Your prompt is not handled by a computer. It is handled by a rack.

The current standard unit is the NVL72: 72 GPUs wired together with NVLink, a very fast, very short-range interconnect that lets the chips read each other's memory quickly enough that software can treat all 72 as a single enormous processor. This matters because modern models are too large to fit in one chip's memory. They are split across many, and the splitting only works if the chips can talk faster than they can think.

One of these cabinets draws around 120 kilowatts — about what forty homes pull — and costs roughly $3 million. A single campus may hold thousands of them.

The next generation raises the stakes. NVIDIA's Vera Rubin NVL144 platform ships in the second half of 2026, with each GPU drawing 1.8 to 2.3 kilowatts against roughly 1 kW for the current Blackwell generation. At those densities liquid cooling is no longer an optimization; it is mandatory. Air simply cannot move the heat.

NVIDIA is not alone in the rack, either. Google's TPUs, Amazon's Trainium chips, and AMD's MI-series now serve a meaningful share of AI inference, and the mix is shifting as inference — not training — becomes the dominant workload.

[ 06 · Inside the Model: Why AI Answers Word by Word ]

This is the layer that explains the thing everyone notices and nobody explains: why the answer arrives in pieces.

Generating a response happens in two very different phases.

The first is prefill. The model reads your entire prompt at once, in parallel, across thousands of processing units. Whether you sent ten tokens or ten thousand, this phase is fast and compute-hungry, and it is over before you notice.

The second is decode, and decode is stubbornly sequential. The model produces one token at a time. To produce each one, it must run your context through the model's full set of weights — hundreds of billions of numbers — and those numbers have to be pulled out of memory and streamed into the chip. Then it appends that token to the context and does the entire thing again for the next one.

Two recent shifts make this dramatically more expensive. Reasoning models generate thousands of hidden tokens before they write anything you see, working through the problem privately; every one of those hidden tokens costs a full pass through memory. And agents — models that search, call tools, read results, and try again — multiply the number of round trips through the whole stack for a single request from you.

A one-line question can quietly become a great deal of computation.

[ 07 · The Chip: 336 Billion Transistors Fed by Stacked Memory ]

Zoom into one GPU and you find a slab of silicon roughly the size of a postage stamp, manufactured by TSMC on a 3-nanometer process, carrying about 336 billion transistors. That is more than forty switches for every human being alive, on one piece of silicon, each one flipping billions of times a second.

Surrounding the die are towers of stacked memory. NVIDIA's Rubin GPU carries 288 GB of HBM4 across eight vertical stacks, delivering up to 22 terabytes per second of bandwidth — the equivalent of reading a full one-terabyte laptop drive twenty-two times over, every second.

That bandwidth number is the one to remember, because it — not raw arithmetic speed — is the real speed limit of AI inference. Every token you receive requires the model's weights to travel from memory into the processor. The chip is almost always waiting on memory, not on math. This is why the memory stacked around the die matters as much as the die itself, and why HBM supply has become its own global bottleneck.

[ 08 · The Fab: The Most Complex Buildings Humans Construct ]

These chips are made in perhaps the most demanding factories on Earth, and overwhelmingly by one company: TSMC.

A leading-edge fab costs $20–50 billion to build. Inside, the air is filtered to a cleanliness far beyond a surgical theatre, because at these dimensions a single speck of dust is a boulder — large enough to destroy a chip whose features are smaller than a virus. Humans in the cleanroom wear full-body gowns mostly to protect the wafers from them.

The wafers themselves rarely touch human hands. They travel in sealed pods called FOUPs, ferried between machines by robots running on overhead rails, in a choreography that never stops.

The process takes about three months and thousands of individual steps. A wafer is coated, exposed to patterned light, etched, layered, polished, measured, and sent back around — dozens of times over, building up more than seventy circuit layers, each aligned to the ones beneath it within a few atoms. A chip that ships this winter began as bare silicon in the autumn.

[ 09 · The EUV Machine: Where Every AI Answer Begins ]

At the center of the fab sits the machine that makes the whole pyramid possible, and there is exactly one company in the world that builds it: ASML, in the Netherlands.

Extreme ultraviolet lithography works like this. A stream of molten tin droplets falls through a vacuum chamber. A high-power laser strikes each droplet — 50,000 times per second — vaporizing it into a plasma hot enough to emit light at a wavelength of 13.5 nanometers. That light is thousands of times finer than a human hair, and it is the reason transistors can still be shrunk.

EUV light is absorbed by essentially everything, including glass and air, so it cannot be focused with lenses. Instead it bounces off a series of mirrors made by Zeiss, polished to a smoothness of less than 0.05 nanometers — flat enough that, scaled up to the size of a country, the tallest bump would be a fraction of a millimetre. The mirrors project the circuit pattern down onto the wafer.

Roughly 70 of these machines are built per year, at $200–400 million each. Fewer than six a month, for the entire planet. The next generation, High-NA EUV, costs around $400 million per unit and is now installed at Intel, TSMC and Samsung.

Every frontier AI chip in existence was printed by one of these. Every answer you have ever received from a chatbot traces back through this machine.

[ 10 · The Two Bottlenecks ]

Stand back from the whole stack and the shape of the constraint becomes clear. Two things govern how much artificial intelligence the world can actually run, and neither of them is software.

EUV sets the ceiling on how many advanced chips can exist. Around seventy machines a year, from a single supplier, feeding a handful of fabs. You cannot buy your way past it in a quarter; the machines take years to build and years to install.

The electric grid sets the ceiling on how many of those chips can be switched on. A chip in a warehouse with no interconnection is an asset, not a capability. With queues running four to seven years and transformers in short supply, power is now the thing that decides how fast this all goes.

Everything else — the model architectures, the training runs, the products, the valuations — sits on top of those two numbers. When you watch a paragraph appear one word at a time, you are watching the visible end of a chain that terminates in tin droplets and substations.

[ Frequently Asked Questions ]