GPU Architecture · Reference
Pascal to Rubin and beyond — what changed each generation, why each is worth paying more for, and how the datacenter parts map to the GeForce cards you know.
NVIDIA quotes peak throughput along three axes that are easy to conflate — getting them wrong is the #1 source of error on these parts:
1. Dense vs. sparse. Every Hopper/Blackwell low-precision tensor peak on a datasheet is quoted with 2:4 structured sparsity (Lesson 11's sparsity feature). Dense = exactly half. NVIDIA leads with the sparse number.
2. Per-die vs. per-GPU. A Blackwell "GPU" is two dies fused into one package. "20 PFLOPS FP4 per GPU" is a 2-die figure; per-die it's ~10. Cross-gen "per-GPU" comparisons quietly compare one Hopper die to two Blackwell dies.
3. Per-GPU vs. per-rack. NVL72/NVL144 exaFLOP figures are ×72 GPUs. Never compare a rack exaFLOP to a chip petaFLOP.
Evidence tags: documented NVIDIA whitepaper/datasheet/keynote · reported reputable press citing NVIDIA · rumor unverified — do not repeat as fact. Numbers verified against NVIDIA primary sources where noted; roadmap beyond Blackwell is where confidence drops.
The one table the whole card hangs on — datacenter architecture, its flagship silicon, and the consumer GeForce line it corresponds to. Note where the names match and where they diverge.
| Arch | Year | Flagship datacenter | Consumer / GeForce | Process | Memory |
|---|---|---|---|---|---|
| Pascal | 2016 | Tesla P100 | GTX 10-series (same name) | TSMC 16 nm | HBM2 |
| Volta | 2017 | Tesla V100 | — (none; Titan V only) | TSMC 12 nm | HBM2 |
| Turing | 2018 | T4 | RTX 20-series (same name) | TSMC 12 nm | GDDR6 |
| Ampere | 2020 | A100 | RTX 30-series (same name, diff. fab) | TSMC N7 (A100) | HBM2e |
| Hopper | 2022 | H100 · H200 | Ada Lovelace RTX 40 (diverged) | TSMC 4N | HBM3 → HBM3e |
| Blackwell | 2024–25 | B200 · GB200 · B300 | RTX 50-series (same name, diff. silicon) | TSMC 4NP | HBM3e |
| Rubin | 2026 reported | VR200 / Rubin NVL144 | not yet announced | TSMC N3 | HBM4 |
| Rubin Ultra | 2027 reported | Rubin Ultra NVL576 | — | N3-class | HBM4e |
| Feynman | 2028 rumor | Feynman (TBD) | — | ~A16 rumor | "custom HBM" |
People most often get this wrong, so it's worth stating plainly. The datacenter and consumer lines have related but not identical naming across time:
The same flagships as specs, so you can watch the levers move — process shrinks, transistors and memory climb, NVLink widens, and the signature tensor format drops in precision each generation.
| Flagship | Process | Transistors | Memory | NVLink | Signature math | TDP |
|---|---|---|---|---|---|---|
| P100 | 16 nm | 15.3 B | 16 GB HBM2 · ~0.72 TB/s | 1.0 · 160 GB/s | FP16 (no tensor core) | 300 W |
| V100 | 12 nm | 21.1 B | 16–32 GB HBM2 · 0.9 TB/s | 2.0 · 300 GB/s | FP16 tensor (1st gen) | 300 W |
| T4 | 12 nm | 13.6 B | 16 GB GDDR6 · 0.3 TB/s | none | INT8 / INT4 tensor | 70 W |
| A100 | N7 | 54.2 B | 40–80 GB HBM2e · ~2.0 TB/s | 3.0 · 600 GB/s | TF32 · BF16 · FP64 tensor · 2:4 sparsity | 400 W |
| H100 | 4N | 80 B | 80 GB HBM3 · 3.35 TB/s | 4.0 · 900 GB/s | FP8 + Transformer Engine | 700 W |
| H200 | 4N | 80 B | 141 GB HBM3e · 4.8 TB/s | 4.0 · 900 GB/s | FP8 (same compute as H100) | 700 W |
| B200 | 4NP | 208 B (2 dies) | 180–192 GB HBM3e · 8 TB/s | 5.0 · 1.8 TB/s | FP4 / FP6 microscaling | 1000–1200 W |
| B300 (Ultra) | 4NP | 208 B (2 dies) | 288 GB HBM3e · 8 TB/s | 5.0 · 1.8 TB/s | FP4 (~1.5× B200) | 1400 W |
| Rubin (VR200) reported | N3 | — | ~288 GB HBM4 · ~20 TB/s | 6.0 | NVFP4 (HBM4 era) | — |
Low-precision figures elsewhere on this card are the sparse peaks NVIDIA leads with; dense is half. SXM/liquid-cooled variants shown where they differ from PCIe/air-cooled.
The heart of the question: at datacenter scale nobody buys $/GPU — they buy $/token-served or $/training-run (the economics section closes the card). Each generation earns its price premium with essentially one headline capability.
Tesla P100 (GP100) · 3,584 CUDA cores · FP64 5.3 TFLOPS
The first HBM2 and the first NVLink GPU. Still a classical FP64-HPC-plus-FP16 chip — no tensor cores yet. The pitch was memory bandwidth and multi-GPU scaling: ~3× Maxwell's bandwidth and a 160 GB/s GPU-to-GPU link (vs. ~16 GB/s PCIe) that let an 8-GPU node act as one tightly coupled machine.
First HBM2 + NVLink — bandwidth and multi-GPU FP64/HPC scaling.
Consumer sibling: GTX 10-series (GP102/104/106) — GDDR5X, no HBM, no NVLink, deliberately weak FP64 (1:32). GP100 was compute-only, never a GeForce card.
Tesla V100 (GV100) · 5,120 CUDA cores · 640 first-gen Tensor Cores · 125 TFLOPS FP16 tensor
The generational break. The first Tensor Cores (Lesson 11) turned deep learning from an FP32-CUDA-core workload into a dedicated-datapath one: ~125 TFLOPS of FP16-in/FP32-accumulate matrix math vs. ~15.7 TFLOPS FP32 — roughly 12× the DL-training throughput of Pascal. Mixed-precision training became the industry default because of this chip. FP32/FP64 only rose ~1.5×; the tensor cores alone justified the price.
First Tensor Cores — ~12× deep-learning training; the reason mixed-precision exists.
Consumer sibling: essentially none. NVIDIA skipped Volta for GeForce (gaming went Pascal → Turing); only the $2,999 Titan V existed. The first tensor cores most developers touched were in V100s.
T4 (TU104) · 320 second-gen Tensor Cores · 130 INT8 TOPS · 70 W
Not a peak-FLOPS play — an efficiency play. Turing's tensor cores added INT8/INT4 paths built for quantized inference, and the T4 packed 130 INT8 TOPS into a 70 W, low-profile, passively cooled slot that drops into any mainstream server. You deploy racks of cheap 70 W inference cards instead of 300 W accelerators — far better perf/watt and perf/$ for scale-out inference and video.
INT8/INT4 tensor at 70 W — best perf/watt for scale-out inference.
Consumer sibling: RTX 20-series — the first consumer RT and tensor cores. The T4 is literally the datacenter cut of the RTX 2080's TU104 die (ECC GDDR6, passive cooling, inference-tuned power).
A100 (GA100) · 6,912 CUDA cores · 432 third-gen Tensor Cores · TF32 312 TFLOPS (sparse)
A three-in-one value argument. TF32 gave existing FP32 models a ~10–20× tensor speedup with no code changes (the hardware truncates the mantissa going in). 2:4 structured sparsity doubles tensor throughput again on pruned models. And MIG partitions one A100 into up to seven hardware-isolated GPUs (Lesson 9), turning a single expensive accelerator into a fleet and attacking cost-per-job directly. FP64 Tensor Cores (2.5× V100) extended the value into classical HPC — "one accelerator for every math type."
TF32 + 2:4 sparsity + MIG (+ FP64 tensor) — free FP32 speedup, doubled sparse throughput, one card as seven.
Consumer sibling: RTX 30-series (GA102), but on Samsung 8 nm vs. the A100's TSMC N7. The datacenter die dropped RT cores and added FP64 tensor + HBM + MIG; the consumer die did the reverse.
H100 (GH100) · 132 SMs · 4th-gen Tensor Cores · FP8 3,958 TFLOPS (sparse)
The LLM chip. The Transformer Engine plus FP8 (both E4M3 and E5M2) dynamically manages precision per layer with automatic loss scaling — native hardware+software for training and serving transformers, ~4× the FP8 throughput of A100's best. Hopper also added the plumbing that modern kernels lean on: thread-block clusters and distributed shared memory, the TMA async bulk-copy engine, DPX dynamic-programming instructions, HBM3, NVLink 4.0, and 2nd-gen MIG with confidential computing.
Transformer Engine + FP8 — native LLM precision; buy Hopper to train/serve transformers.
Consumer contemporary: Ada Lovelace RTX 40 (AD102) — a different architecture name and die on the same 4N node. Ada does have FP8 4th-gen tensor cores (used for DLSS/inference); what it lacks is Hopper's datacenter stack — HBM, high-bandwidth NVLink, clusters, TMA, DPX, and datacenter-scale FP8 training.
H200 · same GH100 compute · 141 GB HBM3e · 4.8 TB/s | GH200 · Grace CPU + Hopper
Two ways to sell the same compute die again. H200 is an H100 with only the memory changed — 141 GB HBM3e (+76% capacity) at 4.8 TB/s (+43% bandwidth). LLM inference is memory-capacity- and bandwidth-bound, not compute-bound (weights + KV cache must fit; decode is bandwidth-limited), so bigger/faster memory alone buys ~2× inference with zero change to the math units. GH200 fuses a 72-core Grace Arm CPU to a Hopper GPU over NVLink-C2C at 900 GB/s coherent, making CPU LPDDR5X and GPU HBM one address space (~576 GB) — for models and datasets that exceed HBM.
H200 = memory for inference. GH200 = coherent CPU↔GPU memory at 900 GB/s.
B200 · two dies + NV-HBI 10 TB/s (one GPU) · 208 B transistors · FP4 20 PFLOPS (sparse, per package)
Three step-changes at once. FP4/FP6 microscaling (NVFP4) via 5th-gen Tensor Cores and a 2nd-gen Transformer Engine — a new precision floor that roughly quintuples H100's FP8 throughput. The dual-die package: two reticle-limit dies joined by a 10 TB/s NV-HBI link and presented to software as one GPU — the end of the monolithic era (a shift the AMD vs NVIDIA card contrasts with AMD's earlier, seam-exposed chiplets). And NVLink 5.0 at 1.8 TB/s per GPU, which enables the rack below.
FP4 + dual-die package + NVLink-5 — a step-function in low-precision inference and in how many GPUs act as one.
Consumer sibling: RTX 50-series (GB202) — same name, completely different silicon: a monolithic GDDR7 gaming die (RTX 5090 = 32 GB GDDR7, PCIe, no NVLink), not the dual-die HBM3e datacenter package. Both are 5th-gen Blackwell and both do FP4 (DLSS 4).
GB200 NVL72 · 72 Blackwell GPUs + 36 Grace CPUs · one NVLink domain · 1.44 EF FP4 (sparse)
The unit of sale becomes the rack. NVL72 wires 72 GPUs into a single 130 TB/s NVLink domain with 13.4 TB of HBM3e, liquid-cooled at ~120 kW, so the whole rack behaves as one coherent accelerator for trillion-parameter models. Blackwell Ultra (B300/GB300) then does the H200 trick to Blackwell: +50% memory (288 GB HBM3e) and ~1.5× FP4, again a memory-and-inference play for 300 B+ parameter serving.
GB200 NVL72 = rack-as-the-unit (72-GPU NVLink domain). GB300 = +50% memory + ~1.5× FP4.
Since Hopper, NVIDIA pairs each GPU generation with an in-house Arm datacenter CPU over coherent NVLink-C2C. Don't confuse the two tracks:
Everything below is roadmap. Architecture names, the annual cadence, and headline rack numbers are NVIDIA-announced; specific process nodes, bandwidths, and anything about cancellations are press-reported or rumor. Treat accordingly.
VR200 / Rubin NVL144 · HBM4 · NVLink 6 · TSMC N3-class
Successor to Blackwell, named for astronomer Vera C. Rubin. NVIDIA's NVL144 rack slide claims 3.6 exaFLOPS FP4 inference / 1.2 exaFLOPS FP8 training, ~3.3× a GB300 NVL72, with 75 TB of rack "fast memory." Counting nuance: "NVL144" does not mean the rack doubled — from Rubin on, NVIDIA counts reticle-sized dies, not packages. NVL144 = 144 dies across the same 72-package footprint as NVL72. rumor memory vendors reportedly missing the ~22 TB/s HBM4 target, landing nearer ~20 TB/s.
HBM4 + NVLink 6 + a full-node process shrink — the next capacity/bandwidth/scale step.
Rubin Ultra NVL576 · HBM4e · ~1 TB per package
NVL576 = 576 dies (4 reticle-sized dies per package × 144 packages), ~100 PFLOPS FP4 per package, ~15 EF FP4 inference per rack (~14× GB300 NVL72), on the new "Kyber" rack with vertical compute blades. rumor a July 2026 single-source report claimed the four-die design was cut over packaging limits — this conflicts with NVIDIA's roadmap and is unconfirmed.
HBM4e + a 4-die package — density for the largest training runs.
Feynman GPU (specs TBD) · paired with the Rosa CPU
On NVIDIA's public roadmap for 2028 under an annual-cadence commitment, named for Richard Feynman, with "custom HBM" and (rumored) true 3D logic-die stacking on a ~TSMC A16 node. Little is officially published; everything specific here is rumor.
3D die stacking + custom HBM — a packaging shift, if it lands as described.
The recurring puzzle in the question — how each generation is "more attractive even at a higher dollar price" — resolves once you see what the buyer is actually optimizing. Hyperscalers buy $ per unit of useful work (tokens served, training runs completed), not $ per GPU. A part that costs 2× but does 3–5× the work per rack is cheaper on the metric that matters. Four compounding levers do it every generation:
One-line version: the sticker price is the smallest term. Power envelope, memory capacity, and interconnect scale decide how many tokens a rack produces — and that is what the buyer is really purchasing.
How to use this card. The generations are one story told with a shifting bottleneck: Pascal solved bandwidth, Volta added the Tensor Core, Turing made inference cheap, Ampere made it flexible, Hopper made it native to transformers, and Blackwell moved the unit from the chip to the rack. When you read a new NVIDIA spec, run it through the three-axis filter at the top — dense or sparse? one die or two? chip or rack? — and most of the marketing resolves into a real number you can compare.