System Design: A Pound of Flesh

Open ten system design writeups from ten unrelated companies, a rideshare dispatcher, a social feed, a payments ledger, an e-commerce search page, and the same names appear in the diagrams like recurring characters. Redis in the corner. Kafka running down the middle. Cassandra or Elasticsearch or BigQuery hanging off the side.

The standard explainer treats each of these as a product to be learned. Here is Redis, it is fast. Here is Kafka, it is a log. That framing misses the only interesting question, which is why the same tools keep showing up together in systems that have nothing in common. Rideshare and payments do not share a domain. They share a database.

Almost every one of these systems started as an application talking to a relational database. Postgres, or something shaped like it. And Postgres is a phenomenal deal. Transactions, joins, constraints, durability, one copy of the truth, all in one box. There is no shortage of "just use Postgres" articles for this reason.

Then the system grows and Postgres hits a wall. Not one wall, several distinct walls, each with its own failure signature. For each wall the industry has settled on one escape hatch. That is the recurrence. Six tools, because there are roughly six walls.

But every escape hatch charges a toll, and the toll is always a guarantee. Postgres gave you a bundle of promises. Each specialized tool does one thing better than Postgres by breaking one of those promises on purpose. You do not get the speed and keep the durability. You do not get the write scale and keep the consistency. The bond is signed the moment you add the box to the diagram. The flesh is collected later, usually at 3 a.m.

The six tools are not six equal moves. They are three kinds of move. Two of them replace Postgres on the hot path. Three of them project your data into a new shape beside Postgres. One of them exists to move data between all the copies you just created. Replace, project, move. Take the bargains in that order and the diagram stops looking like a parts catalog and starts looking like one decision made six times.

I. Replace

The first kind of escape puts a new store directly on the read or write path, doing a job Postgres used to do. These are the true replacements, and they pay the classic toll. A core ACID guarantee, given up outright.

Redis, the wall is latency

The first wall is the oldest one. Your read path is slow, and worse, it is repetitively slow. The same user profile, the same product page, the same session lookup, built from the same joins, thousands of times a minute. Postgres answers each one correctly, in a few milliseconds on a good day. Your p99 remembers the bad days.

So you reach for Redis. Everything lives in RAM, the data structures are the API, and reads come back in microseconds. The wall falls immediately.

The flesh is durability. Redis holds your data in memory, and if the box restarts, you lose everything since the last snapshot. There are persistence options, RDB snapshots and an append only file, but tune them for real durability and you have rebuilt a slow database on top of a fast one. Real deployments run Redis in the configuration that made it worth deploying. The bargain works because you only put data there that you can afford to lose. Caches, sessions, counters, things Postgres can regenerate. The moment somebody stores the only copy of something in Redis, the toll comes due with interest. Every Redis horror story is the same story. Someone forgot which promise they had sold.

Cassandra, the wall is write scale

The second wall is harder, because you cannot cache your way out of it. Writes. One Postgres primary accepts all of them, and a primary is one machine. You can buy a bigger machine for a while. Vertical scaling is a real strategy right up until the invoice or the hardware ceiling, whichever arrives first. Read replicas do not help, since every replica replays the same write stream. When your ingest is a firehose, telemetry, events, messages, sensor data, the single writer design is the wall.

So you reach for Cassandra. A masterless ring where every node accepts writes, and adding capacity means adding machines. Multi region writes, no single point of failure, linear scale. The wall is gone.

The flesh is consistency, and the relational model goes with it. Cassandra is honest about the trade. Consistency is tunable by design, and the settings that deliver the write throughput you came for are the ones where a read may not see the write that came before it. Last write wins, reconciled by timestamps, eventually. You also lose joins and ad hoc queries. You denormalize everything and design your tables backwards from your queries. Postgres let you ask any question of one copy of the truth. Cassandra gives you planetary write scale on the condition that you know every question in advance and can tolerate the answers being briefly wrong. For a message timeline or a sensor feed, that is a fine bond to sign. For an account balance, the creditor is standing right there.

II. Project

The second kind of escape is different in kind, and almost nobody names the difference. Elasticsearch, the vector database, and BigQuery do not replace Postgres. Postgres stays exactly where it is, still the source of truth, still taking the writes. These tools sit beside it, each holding a copy of your data reshaped for one access pattern. An inverted index, an embedding space, a columnar layout. They are projections. The same truth, arranged so that one specific question becomes cheap.

That difference matters because projections all charge the same toll, and it is a new one. Not durability, not consistency within the store. The flesh is the single source of truth itself. The moment the first projection comes online, "the data" stops being a place and becomes a claim that several systems have to agree on. Hold that thought. The bill arrives in Part III.

Elasticsearch, the wall is search

Postgres can search text. LIKE '%term%' will scan its way to an answer, and full text search with tsvector is genuinely serviceable. But the wall is not whether it can match a string. The wall is relevance. Fuzzy matching, typo tolerance, stemming, ranking results by how good a match is rather than whether one exists. Search as users mean the word. Postgres holds your data in rows, and rows are the wrong shape for that question.

Elasticsearch holds the same data as an inverted index. Every term points at every document containing it, with BM25 scoring, analyzers, and aggregations on top. Same truth, reshaped, and "red running shoes, roughly spelled" becomes a millisecond question.

The flesh is freshness. Documents become searchable a refresh interval after they are written, not at commit. But the real cut is the projection toll. Your products table now exists twice, and the copies drift the moment your sync pipeline hiccups. Search results pointing at deleted products. Prices a save behind reality. Nothing in either store is broken. The agreement between them is.

The vector database, the wall is meaning

The newest wall, and the one I want to be honest about, because it is the weakest of the six. Keyword search matches the words a user typed. Increasingly the question is about what they meant. Semantic retrieval, similarity, RAG over documents. The query "how do I cancel" should find the passage titled "Ending your subscription," and no inverted index will make that connection, because the connection is not in the characters. It is in the meaning.

Here is the honest part. Postgres already does this. The pgvector extension is one CREATE EXTENSION away, it stores embeddings in a regular column, and it handles ANN search well into the millions of vectors. For most applications, including most RAG applications, that is the whole requirement. This is the one door on the list that people walk through before hitting any wall. They pay the projection toll, a second copy of the corpus, a sync pipeline, drift, and get nothing for it that a Postgres extension would not have given them.

The wall becomes real at scale. Tens or hundreds of millions of vectors. Heavy filtered search, where you need similarity plus a dozen metadata predicates and the query planner starts to buckle. Index builds that compete with your production workload for memory. That is when a dedicated vector store earns its place, and the move is the same projection move as Elasticsearch. A copy of your corpus in a shape where "similar in meaning" is a cheap question.

The toll has a twist that makes it heavier than the other projections. The projection is lossy and versioned. An inverted index can be rebuilt from the text and you get the same index every time. An embedding space depends on the model that produced it, and when you change that model, and you will, every vector in the store goes stale at once. You do not sync this projection. You re-create it wholesale, and until you do, old points and new points live in incompatible geometries. The single source of truth is gone, and its replacement is not even a faithful copy. It is an interpretation.

BigQuery, the wall is the question that reads everything

The last wall is analytical. "Revenue by region by month for three years" is one question that touches every row you have. Run it on your production Postgres and you will learn why OLTP and OLAP are different acronyms. The row oriented layout that makes "fetch order #4412" instant makes "scan the amount column across two billion orders" a catastrophe. To read one column, you read every column of every row, on the same disks serving your customers.

A reasonable objection here: did we not already add Cassandra, and is Cassandra not a column store? No, and the naming is the trap. Cassandra is a wide column store, which means rows can hold flexible, sparse columns inside a partition. The data is still laid out and fetched by row, by partition key, and Cassandra is worse at analytics than Postgres is. No joins, no ad hoc queries, and a full table scan is the exact operation its design punishes. Wide column answers "get me this partition fast, at any write volume." It cannot answer "sum one column across everything."

Columnar in the BigQuery sense is the opposite layout. Each column stored contiguously, compressed, scannable on its own, spread across enormous parallelism. The billion row aggregate comes back in seconds, and your production database never feels it.

The flesh is that the warehouse is bad at point lookups and transactional updates, which is fine, because Postgres kept those. The real cut, once more, is the projection toll, here at its most visible. An entire ETL or ELT pipeline whose only job is to move truth from the row store to the column store, with the analytics always one pipeline run behind reality. Every company has a Slack thread titled some version of "why the dashboard does not match the database." The answer is always the same. Because you projected, and projections drift.

III. Move

Postgres holds the truth. Redis holds the hot subset. Cassandra took a table or two outright. Elasticsearch holds an inverted projection, the vector store a semantic one, BigQuery a columnar one. One fact, a user changed their address, an order shipped, may now need to be reflected in five places, owned by five systems, each of which you adopted precisely because it does not behave like the others.

How do they agree?

Your first answer will be dual writes. The application writes to Postgres, then writes to Elasticsearch, then invalidates Redis, then... the second write fails, and now two stores disagree and nothing knows it. Dual writes are a distributed transaction you are performing by hand, without the transaction. Your second answer will be batch sync jobs, which work, in the sense that the disagreement is now scheduled.

The answer the industry settled on is the log. Kafka, the sixth tool, and the odd one out, because it is not a store of your data at all. It is a durable, ordered, replayable record of what happened. Every change is an event, appended once, consumed independently by every projection at its own pace. Postgres commits, the change enters the log, usually through change data capture with a tool like Debezium, and Redis, Elasticsearch, the vector pipeline, and the warehouse each read the same stream and update themselves. A new projection next quarter replays the log from the beginning and catches up. Nobody dual writes. Everybody agrees on the order of events, which is the only agreement still available.

Notice what Kafka is for in this diagram. It answers no user query. It searches nothing, caches nothing, aggregates nothing. It exists because you escaped five times. Every bargain in Parts I and II left behind a copy, and the copies must be fed. Kafka is not the sixth wall. Kafka is the pound of flesh you owe on the other five pounds, collected at every commit.

It takes its own cut too. Everything downstream of the log is eventually consistent with the truth, always running a little behind it. You gave up "the data" as a single place. What the log gives back is "the data" as a single history, and everything else in the diagram trails it by some honest number of milliseconds.

The Shape of the Bargain

So the recurrence has a reason. It is always these six because a system that lives long enough hits the same walls in roughly the same order, and each wall has one settled answer, and the answers compound.

You replace what Postgres can no longer do. Redis when the wall is latency, at the price of durability. Cassandra when the wall is write scale, at the price of consistency. You project what Postgres holds into shapes it cannot take, an inverted index, an embedding space, a columnar layout, at the shared price of the single source of truth. And then you move, because five copies must agree about one world, and the log is the only place left where the truth is still singular.