Every Distributed System is Just Solving these 7 Problems
By Manaswini De • Software Engineer & Instructor at CodeKerdos
If you’ve ever looked at a high-level design diagram for Netflix, WhatsApp, Uber, Amazon, or any other internet-scale product, you’ve probably had the same reaction most engineers have in the beginning: everything looks wildly different.
One architecture has Kafka in the middle. Another leans on Redis. One uses sharding, another depends heavily on read replicas, and yet another is split into dozens of microservices. At first glance, it feels as though every company invented a completely different school of architecture.
But after you’ve read enough production postmortems, operated enough services, and watched enough systems fail in creative ways, a simpler pattern starts to emerge: most distributed systems are not solving unique problems. They are solving the same problems under different constraints.
The technologies change. The scale changes. The product surface changes. The priorities change. But the underlying engineering questions remain surprisingly stable.
Great system design is not memorizing tools. It is recognizing recurring problems and choosing the least painful trade-off.
Once you understand those recurring problems, system design diagrams stop feeling like a random collage of technologies. They start looking like logical responses to real bottlenecks, real failures, and real business needs.
Technologies evolve every few years. The seven fundamental problems of distributed systems have remained constant for decades. Build your understanding around the problems, not the tools.
The Seven Problems at a Glance
Before we go deep on each, here is the complete map. Every tool, pattern, and architectural decision you’ll encounter in a distributed system traces back to one or more of these seven categories.
Keep this map in mind as we work through each problem. When you encounter a new technology — say, Apache Flink, or DynamoDB, or Istio — the first question to ask is: which spoke on this wheel does it address?
PROBLEM
01
Scale
Handle more users without rebuilding everything
You have built an e-commerce application. On launch day, 50 users sign up. A single server handles everything gracefully — the application, the business logic, the database. Life is genuinely simple.
Six months later you have five million users. The CPU is pegged at 100%. Memory is exhausted. Response times have climbed from 50ms to 8 seconds. Some users can’t connect at all. The problem is not your code — your code is probably fine. The server simply cannot absorb that volume of concurrent work.
The solution to the scale problem is almost always horizontal scaling — running multiple copies of your application behind a load balancer, rather than buying a progressively bigger single machine (vertical scaling). Horizontal scaling is cheaper, more resilient, and grows proportionally with demand.
But horizontal scaling introduces a requirement: your application must be stateless. If server A handles a user’s login but server B handles their next request, server B must be able to serve them correctly. This means sessions must live outside the application — in Redis, a database, or a distributed cache — not in memory on a single instance.
Real-world examples:
Netflix (thousands of servers across regions)
Amazon (auto-scales during Prime Day)
PhonePe (millions of transactions on festival days)
PROBLEM
02
Latency
Make it feel instant, at any scale
Latency is the time between a user taking an action and your system responding to it. Users are remarkably sensitive to it. Research consistently shows that a 100ms delay reduces conversion rates measurably. At 400ms, users start noticing. At 1 second, many abandon the interaction entirely.
At scale, the latency problem becomes acute. If every Instagram feed load required fetching from a database on disk, serving millions of users would require millions of disk reads per second. Disks are orders of magnitude slower than memory. The numbers simply don’t work.
Caching is the primary weapon against latency — keeping frequently accessed data in fast memory (RAM) so that repeated requests don’t re-do expensive work. Redis, Memcached, and CDN edge caches are all answers to the same problem: the data you need is slow to compute or fetch, but you can store the result and reuse it.
Cache design requires careful thought about two things: what to cache and when to invalidate it. A cache that serves stale data is sometimes worse than no cache at all — and cache invalidation is famously one of the two hard problems in computer science.
⚡
The three latency-busting tools: Caching (serve from memory), CDNs (serve from geographically closer servers), and Read Replicas (distribute read queries across multiple DB copies).
Apply them in that order. Caching is cheapest. CDNs follow for static assets. Replicas for DB-bound reads.
Real-world examples:
Instagram (feed and story caching)
YouTube (video metadata and thumbnail CDN)
Amazon (product detail pages cached at edge)
PROBLEM
03 & 06
Availability & Fault Tolerance
Stay up. Absorb failures. Degrade gracefully.
Here is the uncomfortable truth about distributed systems: in a system with enough components, something is always failing. A disk dies. A network switch drops packets. A cloud provider’s datacenter floods. A deployment goes wrong. These are not rare edge cases — at scale, they are daily realities.
The question is never ‘will this fail?’ The question is always ‘what happens to users when it does?’
Availability and fault tolerance are related but distinct. Availability is about the percentage of time the system is operational (99.9% uptime means ~8.7 hours of downtime per year; 99.999% means ~5 minutes). Fault tolerance is about how gracefully the system handles the failures that do occur.
Netflix’s approach to fault tolerance became legendary: if the recommendation service fails, you can still watch your movie. The system loses one feature, not the entire application. This is graceful degradation in action — designing each component to fail independently rather than cascading failure through the whole system.
// Circuit Breaker pattern — stop calling a failing service
@CircuitBreaker(name = "recommendations", fallbackMethod = "getDefaultContent")
public List<Movie> getRecommendations(Long userId) {
return recommendationService.fetch(userId);
}
// Fallback: return curated popular content instead of failing
public List<Movie> getDefaultContent(Long userId, Exception e) {
return contentService.getTrending(); // always available
}
Real-world examples:
Netflix (circuit breakers + chaos engineering)
Stripe (99.9999% uptime SLA for payment APIs)
PROBLEM
04
Consistency
Keep data correct across copies and concurrent writes
Imagine there is one concert ticket left. Two users click ‘Buy’ at the exact same millisecond. Without coordination, both reads return ‘1 ticket available’, both writes succeed, and you have just sold one ticket twice. Your payment system sends two confirmation emails. You have two very unhappy customers and one very difficult refund conversation.
This is the consistency problem. Distributed systems typically maintain multiple copies of data — for redundancy, for geographic distribution, for read scalability. Keeping all of those copies synchronized, especially under concurrent writes, is one of the deepest challenges in the field.
The CAP theorem (Consistency, Availability, Partition tolerance) formalises the core trade-off: in the presence of a network partition, you must choose between consistency (every read reflects the latest write) and availability (every request gets a response, even if the data is slightly stale). You cannot fully guarantee both simultaneously.
Different products make different choices. A bank account balance must be consistent — showing a stale balance that enables overdrafts has real financial consequences. An Instagram like count that updates one second late? Nobody notices, nobody cares. Choosing the right consistency model for your use case is a core architectural skill.
⚖️
Strong Consistency: every read reflects the latest committed write. Required for financial data, inventory, and anything where stale data causes real harm.
Eventual Consistency: reads may return stale data, but the system converges to consistency over time. Appropriate for social feeds, like counts, recommendation scores.
Real-world examples:
Google Pay
PayPal
Stock trading platforms
PROBLEM
05
Concurrency
Handle simultaneous operations without data corruption
The BCCI opens IPL ticket sales at 10:00 AM. Within seconds, 50,000 people simultaneously hit ‘Book’. Every single one of them is trying to read the same inventory count and write to the same database row. Without concurrency control, the results are predictable: duplicate bookings, phantom tickets, and a customer service team that works overtime.
Concurrency differs from consistency in an important way. Consistency is about keeping multiple copies of data aligned. Concurrency is about handling multiple simultaneous operations on the same data — often on the same single database row — without producing incorrect results.
The four main concurrency strategies each make a different trade-off between safety and throughput:
- Pessimistic locking: Lock the row before reading it. Safe, but reduces parallelism. Good for high-conflict scenarios (ticket booking, seat reservation).
- Optimistic locking: Read freely, but include a version number in the write. Fail and retry if the version changed. Good for low-conflict scenarios.
- Atomic operations: Use database-level Compare-And-Swap (CAS) or atomic decrements. No separate lock needed. High throughput.
- Distributed locks: Use Redis or ZooKeeper to coordinate access across multiple application nodes. Required when a single DB lock isn't enough.
-- Optimistic locking: version column prevents lost updates UPDATE inventory SET quantity = quantity - 1, version = version + 1 WHERE product_id = 42 AND quantity > 0 AND version = :expected_version; -- fails if someone else already wrote -- If 0 rows updated: another user got there first. Retry or notify.
Real-world examples:
BookMyShow (seat locking during checkout)
Flipkart Big Billion Day (inventory atomic ops)
Trading platforms (optimistic locking on order books)
PROBLEM
07
Observability
See inside the system — especially when it's breaking
It is 2:07 AM. On-call engineers are woken by alerts. P99 latency has spiked from 120ms to 4.2 seconds. Customers are complaining. The system is composed of 40 microservices. Where do you even start?
Without observability, the answer is: guesswork. You SSH into servers and tail logs. You run queries you think might reveal the bottleneck. You disable components one by one hoping to isolate the issue. In a distributed system with dozens of services and thousands of nodes, this approach can take hours — hours during which users are suffering and revenue is being lost.
Observability is built on three pillars, each answering a different question about what is happening inside your system:
- Logs (What happened?) Timestamped records of discrete events. ‘User 42 attempted checkout at 02:03:14. Payment gateway timed out.’ Logs are the narrative of your system.
- Metrics (How often? How much?) Numerical measurements over time. Error rate, P99 latency, requests per second, memory usage. Metrics tell you when something changed.
- Traces (Where did time go?) A trace follows a single request through every service it touches, with timing at each hop. Traces tell you which service is responsible for latency.
In the 2 AM scenario above, a distributed trace would immediately show: the API Gateway took 12ms, Redis cache returned in 8ms, but the database query on service ‘order-history-svc’ took 3,940ms. Root cause identified in thirty seconds, not two hours.
🔭
“You can’t fix what you can’t see.” — a maxim so widely repeated in distributed systems that its original author is unknown.
Observability is not an afterthought. It is the feature that makes all other features debuggable. Build it in from day one.
Real-world examples:
Netflix (Atlas metrics + Edgar tracing)
Uber (Jaeger distributed tracing)
Airbnb (Datadog metrics + PagerDuty alerting)
The Same Seven Problems — Different Priorities
Now let’s look at the real-world payoff of this mental model. The following matrix shows how five major distributed systems prioritise these problems. Notice that no company scores equally on all seven — the priorities are shaped entirely by their core product.
Why does Google Pay prioritise Consistency over Scale? Because a payment that debits twice costs more than a payment that takes 200ms longer. Why does Netflix weigh Availability so heavily? Because a streaming service that returns an error is worse than one that suggests yesterday’s trending shows while the recommendation engine restarts.
The product’s core value proposition determines which problems are critical and which are acceptable to trade off. Architecture is the expression of those trade-offs in code and infrastructure.
A New Way to Read Any Architecture Diagram
With these seven problems in your mental toolkit, you can reverse-engineer any system design diagram. The approach is simple — stop asking about the technology and start asking about the problem.
| ❌ Don’t ask this | ✅ Ask this instead |
|---|---|
| “Why is Redis here?” | “Which problem is Redis solving here?” |
| “Why Kafka?” | “What failure mode does Kafka prevent?” |
| “Why multiple databases?” | “Which bottleneck does this extra DB address?” |
| “Why a service mesh?” | “What concurrency or routing problem does this solve?” |
| “Why read replicas?” | “Which latency or scale problem is this targeting?” |
When you ask the right question, the technology choice becomes obvious. Redis appears wherever low-latency data retrieval matters. Kafka appears wherever you need to decouple services or guarantee message delivery through partial failure. Read replicas appear wherever read scale or read latency is the bottleneck. Every component is a logical answer to one of seven questions.
Technology → Problem Reference Table
Use this as a cheat sheet during system design interviews, architecture reviews, or when evaluating a new technology. The key question is always: which problem does this solve, and is that the problem I actually have?
How Experienced Engineers Actually Think
There is a consistent difference in how engineers at different experience levels approach system design conversations.
The senior engineer’s approach isn’t just philosophically superior — it produces demonstrably better architectures. Components chosen to solve specific, understood problems are easier to reason about, easier to change, and easier to remove when the problem evolves. Components chosen because they ‘seem like good practice’ become permanent fixtures nobody wants to touch.
Conclusion: The Problem Is Always the Starting Point
The next time you encounter a system design diagram — in an interview, in a conference talk, in your own team’s architecture review — try this exercise. Point at each component and ask: which of the seven problems is this solving? In most well-designed systems, you’ll get a clear answer for every box.
And the next time you are designing a system yourself, resist the urge to start with the technology stack. Start with the problems. Is latency the primary concern? Start with caching. Is data correctness critical? Design for consistency first. Are you expecting bursts of concurrent writes? Solve concurrency before you solve anything else.
The companies we admire — Netflix, Uber, WhatsApp, Google Pay, Amazon — didn’t become engineering marvels because their engineers had a longer list of technologies. They became engineering marvels because their engineers had a clearer understanding of the problems they were solving.