System Design

System Design From Scratch — Floor 0: Why Anything Is Fast or Slow

The first post in a series where I build system-design intuition from the ground up. No jargon dumped on you up front — we start with an everyday situation, reason it out, and only then attach the technical name.

The premise

Most system-design material throws vocabulary at you — latency, durability, consistency, caching — and expects you to nod along. I've been learning it the other way around: take a scene from daily life, work it out in plain language, and then staple the technical term onto the thing you already understood.

It turns out most of these "scary" words are just labels for intuitions you already have.

This is Floor 0. Before networks, before databases, before anything distributed, there's one question sitting underneath all of it:

Why is anything fast or slow?

Get this floor right and a surprising amount of the rest becomes obvious.

A question from the kitchen

You're cooking, and mid-recipe you need one ingredient. It could be in one of four places:

(a) already in your hand
(b) in the fridge, across the kitchen
(c) at the corner shop, 5 minutes away
(d) shipped from a farm in another country

Two questions.

First, rank them fastest to slowest. Easy: a, b, c, d.

Second — and this is the one that actually matters — how much slower is each step than the one before it? A little slower, or a lot?

The honest answer is: wildly slower. Not 2× or 5× per step. More like 100× to 1000× at every jump. And that single fact explains most of system design.

The four places are the four places a computer keeps data

Those four spots map almost perfectly onto where a computer actually stores things. Here's the trick to feel the gaps: pretend grabbing the ingredient from your hand took 1 whole second, then scale everything else up by the same factor.

The latency hierarchy, scaled to human time: CPU cache 1 second, RAM 2 minutes, disk 1 day, network 3 years

Sit with that bottom row. Reaching across the network is, in human terms, a three-year wait compared to the one-second reach into cache. That's not "a bit slower." That's the difference between now and the next World Cup.

Now the two labels, stapled onto feelings you already have:

Latency = the time it takes to go get the data. High latency = long wait. That word that sounded intimidating? It just means "how far away is the ingredient."
The whole stack — cache → RAM → disk → network — is the memory hierarchy. Its iron rule: each step down is dramatically slower, but also bigger and cheaper. Your hand holds one thing; another country holds everything. Fast-and-small at the top, slow-and-huge at the bottom.

Almost every performance decision in a real system is secretly a question about which shelf the data lives on.

The trick everyone uses: caching

Here's a natural follow-up. Suppose you need an ingredient that lives on the three-year shelf (another country), and you need it hundreds of times a night. Making the full trip every single time is insane.

The obvious real-life move: buy a bunch and keep it in the fridge.

Congratulations — you just invented the single most important pattern in system design. Its name is caching.

Caching = keep a copy of the slow, far-away data in a closer, faster place, so you skip the long trip most of the time.

Your fridge stash is a cache. That's the whole idea. Two pieces of vocabulary come with it, and you already understand both:

A cache lookup: check the fridge first — a hit is fast, a miss means the slow trip to the farm and then stashing a copy

You need it and it's in the fridge → grab it → a cache hit (the fast path).
It's not in the fridge, so you make the trip → a cache miss (the slow path).

And here's the magic, in the numbers you already have: without a cache, every lookup is a three-year trip. With a cache, the first one costs three years, but the next hundred cost two minutes each. That one trick is how systems serve millions of requests quickly even though the data physically lives somewhere slow. A huge chunk of real-world "performance" is just turning misses into hits.

Every trick has a price

Reach for a pattern and you always pay for it somewhere. Finding that price is the whole skill. Caching has two famous ones.

1. The copy goes stale. Your fridge veg spoils, or the farm ships a new-and-better batch — but your fridge still holds the old one. Next time you grab it, you get an out-of-date copy.

The farm has a fresh batch while the fridge still holds the old copy — two copies that disagree is the consistency problem

That copy is stale, and the problem of knowing when to throw it away and refresh is called cache invalidation. It's so notoriously hard there's a famous industry joke:

"There are only two hard things in computer science: cache invalidation and naming things."

2. The fridge is small. Stock it with things you rarely use and there's no room for what you actually need. Deciding what to keep and what to toss is the eviction problem — the common rule being "throw out whatever you haven't touched in the longest time," which you'll meet later as LRU (least-recently-used).

The bridge to the biggest word of all

Look at the root of that staleness problem: you have two copies of the same thing — the farm and the fridge — and they disagree.

The general problem of "multiple copies of data that are all supposed to agree" has a name, and it's one of the most intimidating words in the whole field:

Consistency.

That's all it means: do the copies agree? A stale cache is inconsistent with its source. And it shows up everywhere copies exist — not just caches, but database replicas, backups, CDNs. Same problem, different costume. We'll give consistency its own full treatment further up the building; for now, it's enough that you've felt it.

What you now own

Everything below came from one kitchen scene, reasoned out first and labeled second:

latency — how far away the data is
the memory hierarchy — cache → RAM → disk → network, each ~100–1000× slower but bigger and cheaper
caching and cache hit / miss — keep a close copy, skip the slow trip
the price of caching — staleness / cache invalidation and eviction (LRU)
the seed of consistency — copies that must agree

That's a real foundation, and none of it required memorizing a definition.

Next: Floor 1

The next floor asks: what can a single machine actually do, and where does it hit a wall? That's where "doing many things at once," and the difference between work that waits and work that computes, live — the birthplace of CPU-bound vs I/O-bound, concurrency, and blocking.

See you on Floor 1.