Lesson 5 of 5 · CUDA's memory abstractions

GPU Architecture · Logical Map

CUDA's Memory Spaces

The five names you write for storage — and the physical tier hiding behind each. This closes the second map.

Lesson 4 gave you the physical tiers: registers, L1/shared SRAM, L2, DRAM. But in code you never write "L2." You write in terms of memory spaces — CUDA's logical names for where a variable lives. There are five, and the whole skill is knowing which physical tier each one lands on — because sometimes the name lies to you.

The five spaces

Here's the complete set, how you declare each, and — the payoff column — which Lesson-4 tier actually backs it:

SpaceYou declare it asWho sees itLives as long asPhysical tier
Registera plain local variable — float x;one threadthe threadregister file · ~1 cyc
Locala local var that won't fit / an indexed arrayone threadthe threadDRAM ⚠ · ~400+ cyc
Shared__shared__ float t[256];one blockthe blockL1/shared SRAM · ~30 cyc
GlobalcudaMalloc + a pointerall threads + hostuntil you free itDRAM · ~400+ cyc
Constant__constant__ float c[64];all threads (read-only)the applicationDRAM + constant cache

Two of these you fully own from earlier: registers (Lesson 2's occupancy story) and shared (the scratchpad slice of L1 you met last exchange). The three that need care are local, global, and constant — and local and constant are where the logical name diverges from the physical truth.

Every space in one kernel

__constant__ float coeff[64]; // constant: read-only, host-set

__global__ void k(float* in, float* out) { // in,out: GLOBAL (pointers)
  __shared__ float tile[256]; // SHARED: per-block scratchpad
  int i = blockIdx.x*blockDim.x + threadIdx.x; // i,acc: REGISTERS
  float acc = 0.0f;
  tile[threadIdx.x] = in[i]; // global → shared (the tiling move)
  __syncthreads();
  acc = tile[threadIdx.x] * coeff[0]; // fast reads: shared + constant
  out[i] = acc; // register → global
}

Notice you never name a tier — you write __shared__ or a bare variable, and the compiler + hardware place it. Same pattern as the compute map: you write logical, the machine supplies physical.

Surprise 1 — "local" memory is not local, and not fast

The single most misleading name in CUDA. Local memory is per-thread (that's the "local" — local to a thread) — but physically it lives out in DRAM, the slowest tier, cached through L1/L2 like any global access. It is not on-chip.

When does a variable land here? Two cases:

Why this matters

A kernel that looks like it uses only fast per-thread variables can secretly be hammering DRAM because of spills or indexed arrays. "Local" sounds fast; it's the opposite. This is precisely the physical↔logical gap your mission is about — the name is logical, the cost is physical.

Surprise 2 — constant memory is DRAM, but broadcast-cached

Constant memory is a small (64 KB total), read-only region the host fills before launch. It physically lives in DRAM too — but it's served through a dedicated constant cache with one special power: broadcast. When all 32 threads of a warp read the same address, the cache serves it in a single fetch, effectively as fast as a register.

The flip side: if threads in a warp read different constant addresses, those reads serialize. So constant memory is superb for values every thread shares (coefficients, parameters) and poor for per-thread-varying data. It's a tier tuned for one access pattern.

Surprise 3 — what makes shared memory fast (and how it stalls): banks

You asked what makes shared memory fast. The answer is banks. Shared memory is split into 32 equal banks (one per lane in a warp), and it can serve one address from each bank simultaneously — 32 values in a single cycle. That parallelism is its speed.

It holds only when the warp's 32 threads hit 32 different banks. Two cases break it:

Conflict-free (fast)
b0b1b2b3b4b5b6b7
2-way conflict (2× slower)
b0b0b2b3b4b4b6b7

Top: each thread hits its own bank → one cycle. Bottom: two threads target the same bank (different words) → the hardware serializes them, halving throughput. That's a bank conflict.

The one exception, again, is broadcast: if all threads read the same word in one bank, there's no conflict — one read serves everyone. Laying out shared arrays so a warp strides across banks (not down one) is a core optimization you'll do later. For now: shared memory is fast by construction, and bank conflicts are how you accidentally give that speed back.

The capstone: both maps, complete

This table is the whole course in one frame. On the left, the two logical hierarchies you write; on the right, the physical machine that runs them. You built every row yourself.

Logical — you writePhysical — the silicon
Kernel / Gridthe whole GPU (SMs fed by the GigaThread Engine)
Blockone SM
Warp (32 threads)one sub-partition's scheduler + 32 lanes (SIMT)
Threadone CUDA-core lane
Register / Local varregister file · (or DRAM, if spilled)
Shared memoryon-chip L1/shared SRAM (per SM)
Global / Local / ConstantDRAM (via L2 + L1 / constant cache)
You've finished the map

Five lessons ago, GPU vocabulary was a pile of loose terms. Now every one of them sits in a two-column structure: you shape the left column; the hardware supplies the right. Fast GPU code is just picking a left-column layout whose right-column reality is cheap — keeping warps converged, keeping data on-chip, keeping DRAM traffic low. That's the entire game, and you can now reason about it.

Check yourself

From memory — one click locks each answer.

GotchaA thread declares float buf[16]; and indexes it with a runtime value. That array most likely lives physically in —

MechanismConstant memory serves a warp fastest — like a register — exactly when the 32 threads —

ConceptShared memory serves a warp in a single cycle precisely when the 32 threads' addresses —

Primary source to read next (≈20 min)

CUDA C++ Programming Guide — §"Memory Hierarchy" and §"Device Memory Accesses." NVIDIA's definition of the memory spaces and their qualifiers (__shared__, __constant__), plus the bank and coalescing rules. The canonical version of everything above. Read it here.

The core arc is complete — you have the full mental model of how GPU code runs and where data lives. Two natural directions from here, both of which you flagged earlier: (1) Optimization — put the maps to work: real tiling, coalescing, occupancy tuning, and the __syncthreads() internals you asked about. (2) Hands-on — fire up your RTX 5080: run deviceQuery to see every number from these lessons on your actual chip, then write and profile a first kernel. Tell me which you want next.

Notes & citations

  1. CUDA C++ Programming Guide, "Memory Hierarchy" / "Variable Memory Space Specifiers" — the five spaces, their qualifiers, scope and lifetime; local memory resides in device memory (DRAM); constant memory is 64 KB served by a per-SM constant cache; shared memory is organized in 32 banks. docs.nvidia.com. See resources.html.
← Lesson 4 ⌂ Home Lesson 6 →