Latency Numbers Every Engineer Should Know (with Intuition)

🔵 Fundamentals · Fresher-friendly → 🟠 senior nuance

Why is reading from memory "fast" and reading from disk "slow" — and how slow, exactly? Great engineers carry a rough mental table of these numbers, because system design is really a series of choices about where data lives, and each place has a wildly different speed. This is the famous "Latency Numbers Every Programmer Should Know," and you don't need to memorise them precisely — just the orders of magnitude. Get those, and half of design becomes intuitive. Let's build the picture.

The units, first

These numbers span tiny fractions of a second, so the units matter:

• Nanosecond (ns) = one billionth of a second (10⁻⁹).
• Microsecond (µs) = one millionth (10⁻⁶) = 1,000 ns.
• Millisecond (ms) = one thousandth (10⁻³) = 1,000 µs = 1,000,000 ns.

Each step up is 1,000×. So "µs vs ms" isn't "a bit slower" — it's a thousand times slower. Holding that scale in your head is the whole skill.

The numbers that matter (rounded)

The intuition: memory vs disk vs network

🟠 A vivid way to feel it — scale everything up so 1 ns = 1 second. Then reading from RAM takes ~2 minutes, an SSD read takes ~1.5 days, a disk seek takes ~4 months, and a round trip across the world takes ~5 years. The gaps are staggering. That's why "just read it from disk" or "just add another remote call" are not free — at scale they dominate your latency.

Why these numbers drive design

🟠 Almost every classic design decision falls out of this table:

• Why we cache: RAM is ~1,000× faster than SSD and ~100,000× faster than disk. Keeping hot data in memory (or Redis) instead of hitting the database is the single biggest speed win.
• Why we minimise network calls: a cross-region round trip (~150 ms) can dwarf all your compute. Chatty designs that make many sequential remote calls feel slow — batch them or fetch in parallel.
• Why data locality matters: keeping services and data in the same data centre (~0.5 ms) versus across the world (~150 ms) is a 300× difference. It's why we use CDNs and regional deployments.
• Why SSDs replaced disks for databases: ~100 µs vs ~10 ms is a 100× win on every random read.

Using them in an interview

🟠 You won't be quizzed on exact nanoseconds, but dropping the right order of magnitude at the right moment is powerful: "That lookup can't hit disk on the hot path — a disk seek is ~10 ms, so at 10,000 QPS we'd fall over; I'll serve it from an in-memory cache at ~100 ns." Or: "Two sequential cross-region calls is ~300 ms of pure network — I'd parallelise them or move the data closer." That's you turning a rough table into concrete design decisions — exactly the instinct these numbers are meant to build.

The interview-ready summary

"I keep the rough latency ladder in mind: CPU cache ~1 ns, RAM ~100 ns, SSD ~100 µs, a same-data-centre round trip ~0.5 ms, disk seek ~10 ms, and a cross-world round trip ~150 ms — each big step is about 1,000×. That's why I cache hot data in memory instead of hitting disk, minimise and parallelise network calls, and keep data close to compute. I don't memorise exact figures — I use the orders of magnitude to decide where data should live." That framing shows you reason quantitatively, which is the whole point."

What to read next

• Capacity Estimation — the other half of back-of-the-envelope math
• Caching Strategies — exploiting the RAM-vs-disk gap
• Latency, Throughput & Availability — what these numbers add up to
• ← The complete System Design guide (hub)

← Resilience patterns · Back to the System Design hub →