The crossover
Inference overtakes training fast
Training GPT-3 used ~1,287 MWh, once. At ~0.3 Wh per query, ongoing inference matches that after only a few billion queries — then keeps running forever after.
Every other sheet here is about the running meter — what each query costs to answer. But before a model answers anything, someone spends hundreds of millions of dollars teaching it. Training is AI's other cost, with the opposite shape: enormous, one-time, and paid up front — yet, spread across a model's life, almost nothing per use.
Paid before the model serves anyone — mostly compute, plus energy and data. A bet that the model will earn it back over its life.
≈ building the factory.
Fractions of a cent each — but multiplied by billions of queries a day, every day, for the model's whole life.
≈ the electricity to run it.
| Model | Est. training cost | Notes |
|---|---|---|
| GPT-4 (2023) | ~$78M | ~2×10²⁵ FLOP — the reference point |
| Llama 3.1 405B | ~$170M | open weights, anyone can run the result |
| Gemini Ultra | ~$191M | ~1×10²⁶ FLOP |
| DeepSeek V3 | ~$6M | the efficiency outlier that rattled the market |
| Frontier 2026 class | ~$200–500M | GPT-5 / Gemini Ultra tier, 10²⁶–10²⁷ FLOP |
| Late-2027 frontier | ~$1–3B | projected — the first billion-dollar runs |
The headline figure is terrifying as a lump sum and trivial once divided. A popular model answers hundreds of billions to trillions of queries before it's retired:
$200,000,000 ÷ ~1 trillion queries ≈ $0.0002 / query (0.02¢)That's typically far less than the inference cost of the same query. So the scary training number is real — but it's a one-time foundation, not what you're paying for when you chat.
Training GPT-3 used ~1,287 MWh, once. At ~0.3 Wh per query, ongoing inference matches that after only a few billion queries — then keeps running forever after.
Across a popular model's life, most of its energy and carbon comes from serving, not building (Google, Meta, AWS estimates) — because training happens once and queries happen forever.
Training budgets climb while per-query cost collapses ~10×/yr — better hardware and algorithms spread a bigger upfront cost over far more, far cheaper queries. See sheet 05 →
Open weights (Llama) and distillation let one expensive training run seed countless cheap downstream models — so the world doesn't pay that $200M again and again.
The eye-watering training figure is a one-time foundation — and amortized across a model's working life it nearly vanishes per query. The cost that actually adds up over time is the one this collection spends most of its pages on: inference, repeated forever. Both numbers are real; they just answer different questions.
Note — Training costs are estimates of the final run's amortized hardware + energy; labs almost never publish actuals, and full program cost (failed runs, research, salaries, data licensing) is much higher. Per-query and lifetime figures are illustrative order-of-magnitude math, not exact accounting.
Sources — Epoch AI, How much does it cost to train frontier AI models? (~2.4×/yr training-cost growth since 2016; frontier training-compute ~4–5×/yr, sustainable to ~2030). Dollar figures — GPT-4 ~$78M, Gemini Ultra ~$191M, Llama 3.1 405B ~$170M — are Stanford HAI AI Index cloud-rental estimates; Epoch's amortized figures run lower. DeepSeek V3 ~$6M final-run compute (lab disclosure; Epoch). 2026 frontier runs ~$200–500M, late-2027 ~$1–3B (industry estimates, 2026). Training-vs-inference lifetime split 60–90% inference (Google, Meta AI, AWS analyses). GPT-3 training ~1,287 MWh / ~552 t CO₂e (Patterson et al.).