Design scalable distributed systems using structured approaches for load balancing, caching, database scaling, and message queues. Use when the user mentions "system design", "scale this", "high availability", "rate limiter", "design a URL shortener", "system design interview", "capacity planning", or "distributed architecture". Also trigger when estimating infrastructure requirements, choosing between microservices and monoliths, or designing for millions of concurrent users. Covers common system designs and back-of-the-envelope estimation. For data fundamentals, see ddia-systems. For resilience, see release-it.
A structured approach to designing large-scale distributed systems. Apply these principles when architecting new services, reviewing system designs, estimating capacity, or preparing for system design discussions.
Start with requirements, not solutions. Every system design begins by clarifying what you are building, for whom, and at what scale. Jumping to architecture before understanding constraints produces over-engineered or under-engineered systems.
The foundation: Scalable systems are not invented from scratch -- they are assembled from well-understood building blocks (load balancers, caches, queues, databases, CDNs) connected by clear data flows. The skill lies in choosing the right blocks, sizing them correctly, and understanding the tradeoffs each choice introduces. A four-step process -- scope, high-level design, deep dive, wrap-up -- keeps the design focused and communicable.
Goal: 10/10. When reviewing or creating system designs, rate them 0-10 based on adherence to the principles below. A 10/10 means the design clearly states requirements, includes back-of-the-envelope estimates, uses appropriate building blocks, addresses scaling and reliability, and acknowledges tradeoffs. Lower scores indicate gaps to address. Always provide the current score and specific improvements needed to reach 10/10.
Six areas for building reliable, scalable distributed systems:
Core concept: Every system design follows four stages: (1) understand the problem and establish design scope, (2) propose a high-level design and get buy-in, (3) dive deep into critical components, (4) wrap up with tradeoffs and future improvements.
Why it works: Without a structured process, designs either stay too abstract or get lost in premature detail. The four-step approach ensures you invest time proportionally -- broad strokes first, depth where it matters.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| New service kickoff | Write a one-page design doc with all four steps before coding | Requirements, API contract, data model, capacity estimate, then implementation |
| Architecture review | Walk reviewers through the four steps sequentially | Present scope, high-level diagram, deep-dive on the riskiest component, open questions |
| Incident postmortem | Trace the failure back through the four-step lens | Which requirement was missed? Which building block failed? What tradeoff bit us? |
Core concept: Use powers of two, latency numbers, and simple arithmetic to estimate QPS, storage, bandwidth, and server count before committing to an architecture.
Why it works: Estimation prevents two failure modes: over-provisioning (wasting money) and under-provisioning (outages under load). A 2-minute calculation can save weeks of rework.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| Capacity planning | Estimate QPS then multiply by growth factor | 100M DAU x 5 actions / 86400 = ~5,800 QPS avg, ~30K QPS peak |
| Storage budgeting | Estimate per-record size and multiply by volume and retention | 500M tweets/day x 300 bytes x 365 days = ~55 TB/year |
| SLA definition | Convert availability nines to allowed downtime | Four nines (99.99%) = ~52 minutes downtime per year |
Core concept: Scalable systems are assembled from a standard toolkit: DNS, CDN, load balancers, reverse proxies, application servers, caches, message queues, and consistent hashing.
Why it works: Each block solves a specific scaling or reliability problem. Knowing when and why to introduce each block prevents both premature complexity and avoidable bottlenecks.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| Read-heavy workload | Add cache-aside with Redis in front of the database | Cache user profiles with TTL; invalidate on write |
| Traffic spikes | Insert a message queue between API and workers | Enqueue image-resize jobs; workers pull at their own pace |
| Global users | Place a CDN in front of static assets | Serve JS/CSS/images from edge; origin only serves API |
| Uneven load | Use consistent hashing for shard assignment | Add a node and only ~1/n keys need to move |
Core concept: Choose SQL vs NoSQL based on data shape and access patterns, then scale vertically first, horizontally (replication and sharding) when vertical limits are reached.
Why it works: The database is usually the first bottleneck. Understanding replication, sharding strategies, and denormalization tradeoffs lets you delay expensive re-architectures and plan growth deliberately.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| Read-heavy API | Leader-follower replication with read replicas | Route reads to replicas, writes to leader; accept slight replication lag |
| User data at scale | Hash-based sharding on user_id | Shard key = hash(user_id) % num_shards; even distribution, each shard independent |
| Analytics dashboard | Denormalize into read-optimized materialized views | Pre-join and aggregate nightly; serve dashboards from the materialized table |
| Multi-region app | Multi-leader replication with conflict resolution | Each region has a leader; last-write-wins or application-level merge |
Core concept: Most systems are variations of a small set of well-known designs: URL shortener, rate limiter, notification system, news feed, chat system, search autocomplete, web crawler, and unique ID generator.
Why it works: Studying common designs builds a mental library of patterns and tradeoffs. When a new problem arrives, you recognize which known design it most resembles and adapt rather than invent from scratch.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| Short link service | Base62 encode an auto-increment ID or hash | https://short.ly/a1B2c3 maps to row in key-value store |
| API protection | Token bucket rate limiter at gateway | 100 tokens/min per API key; refill at steady rate; reject with 429 |
| Social feed | Hybrid fanout: push for normal users, pull for celebrities | Pre-compute feeds for accounts with < 10K followers; merge at read time for celebrity posts |
| Distributed IDs | Snowflake: timestamp + datacenter + machine + sequence | 64-bit, time-sortable, no coordination required between generators |
Core concept: A system is only as good as its ability to stay up, recover from failures, and be observed. Health checks, monitoring, logging, and deployment strategies are not afterthoughts -- they are first-class design concerns.
Why it works: Production systems fail in ways that design diagrams never predict. Operational readiness -- metrics, alerts, rollback plans, and redundancy -- determines whether a failure becomes a minor blip or a major outage.
Key insights:
Code applications:
| Context | Pattern | Example |
|---|---|---|
| Zero-downtime deploy | Blue-green with health check gates | Route traffic to green after health checks pass; keep blue as instant rollback |
| Gradual rollout | Canary deploy with metric comparison | Send 5% of traffic to new version; compare error rate and latency; promote or rollback |
| Failure detection | Liveness and readiness probes | /healthz returns 200 if alive; /ready returns 200 if database connected and cache warm |
| Data safety | Define RPO/RTO and implement accordingly | RPO = 1 hour means hourly backups; RTO = 5 min means automated failover |
| Mistake | Why It Fails | Fix |
|---|---|---|
| Jumping to architecture without clarifying requirements | You solve the wrong problem or miss critical constraints | Spend the first 5-10 minutes on scope: features, scale, SLA |
| No back-of-the-envelope estimation | Over-provision or under-provision by orders of magnitude | Estimate QPS, storage, and bandwidth before choosing components |
| Single point of failure | One component failure takes down the entire system | Add redundancy at every layer: multi-server, multi-AZ, multi-region |
| Premature sharding | Adds enormous operational complexity before it is needed | Scale vertically first, add read replicas, cache aggressively, shard last |
| Caching without invalidation strategy | Stale data causes bugs and user confusion | Define TTL, cache-aside with explicit invalidation on writes |
| Synchronous calls everywhere | One slow downstream service cascades latency to all callers | Use message queues for non-latency-critical paths; set timeouts on sync calls |
| Ignoring the celebrity/hotspot problem | One shard or cache key gets hammered, others idle | Detect hot keys, add secondary partitioning, or use local caches |
| No monitoring or alerting | You find out about failures from users, not dashboards | Instrument metrics, logs, and traces from day one |
| Question | If No | Action |
|---|---|---|
| Are functional and non-functional requirements explicitly listed? | Design is based on assumptions | Write down features, DAU, QPS, storage, latency SLA, availability SLA |
| Do you have a back-of-the-envelope estimate for QPS and storage? | Capacity is a guess | Calculate: DAU x actions / 86400 for QPS; records x size x retention for storage |
| Is every component in the diagram redundant? | Single points of failure exist | Add replicas, failover, or multi-AZ for each component |
| Is the database scaling strategy defined? | You will hit a wall under growth | Plan: vertical first, then read replicas, then sharding with a clear shard key |
| Is there a caching layer for read-heavy paths? | Database takes unnecessary load | Add Redis/Memcached with cache-aside and a defined TTL |
| Are async paths using message queues? | Tight coupling, cascading failures | Decouple with Kafka/SQS for background jobs, notifications, analytics |
| Is there a monitoring and alerting plan? | Blind to failures in production | Define metrics, log aggregation, tracing, and alert thresholds |
| Is the deployment strategy defined? | Risky all-at-once releases | Choose rolling, blue-green, or canary with automated rollback |
This skill is based on Alex Xu's practical system design methodology. For the complete guides with detailed diagrams and walkthroughs:
Alex Xu is a software engineer and the creator of ByteByteGo, one of the most popular platforms for learning system design. His two-volume System Design Interview series has become the de facto preparation resource for engineers at all levels, with over 500,000 copies sold. Xu's approach emphasizes structured thinking, back-of-the-envelope estimation, and clear communication of design decisions. Before ByteByteGo, he worked at Twitter, Apple, and Oracle. His visual explanations and step-by-step frameworks have made system design accessible to a broad engineering audience, transforming what was traditionally an opaque topic into a learnable, repeatable skill.