The Wall.

Computers don't wait for calculation.
They wait for memory.

Scroll to Descend

The Latency Pyramid

SRAM (L1/L2)

GPU CORE

HBM (High Bandwidth)

DRAM (System Memory)

SSD ARRAY

NAND (Flash Storage)

1. SRAM The Hand

Static Random Access Memory. This is the cache directly on the chip.

Latency: <1 nanosecond
Cost: Extremely High ($$$$)
Analogy: The thoughts in your head.

2. HBM The Desk

High Bandwidth Memory. Stacks of DRAM fused right next to the GPU.

Latency: ~10 nanoseconds
Bandwidth: 3TB/s+
Analogy: The open books on your desk.

3. DRAM The Library

System Memory. Massive capacity but requires traveling across the motherboard.

Latency: ~100 nanoseconds
Capacity: Terabytes
Analogy: Walking to the shelves.

4. NAND The Archive

Flash Storage. Persistent but glacial speeds compared to the processor.

Latency: Microseconds (1000x slower)
Capacity: Petabytes
Analogy: Dealing with the bureaucracy in the basement.

This is the bottleneck.

The reason we need HBM4 and CoWoS packaging is to minimize the distance betweenLogic and Memory.

Explore The Transformer →

BACK TO INTELLIGENCE