Building a system that handles millions of users requires different decisions than building one that handles thousands. System design is about making tradeoffs explicit: scalability, availability, consistency, latency, and cost.
TL;DR: A system design reference covering scalability patterns, load balancing, caching, databases, message queues, and distributed system tradeoffs.
Stack: System Design, distributed systems, architecture
Level: Advanced
Reading time: ~30 min
Core reliability concepts
Reliability means your system starts every time, like a dependable car. Load balancers do health checks to route traffic only to healthy instances.
Fault Tolerance means the system keeps functioning even when parts fail, like an airplane flying safely with one engine out. EC2 Auto Scaling replaces failed instances automatically.
Redundancy adds safety nets at every layer. AWS Multiple Availability Zones duplicate critical infrastructure so a single AZ failure doesn’t take you down.
SLO/SLA: An SLO is a measurable internal goal (99.9% uptime). An SLA is a contractual commitment to customers. CloudWatch Alarms and API Gateway throttling help you stay within both.
DDoS
A DDoS attack is like a crowd blocking the entrance to a store, preventing real customers from getting in. AWS mitigations: AWS Shield for automatic protection, WAF for traffic filtering, CloudFront for absorbing edge traffic, and Auto Scaling to absorb volumetric attacks.
Throughput
Throughput is the number of requests a system processes per second. Think of it as lanes on a highway: more lanes (more servers) means more traffic gets through. ELB distributes load, ElastiCache serves cached data without hitting the database, CloudFront offloads static assets, and Auto Scaling adds lanes when traffic spikes.
Single Point of Failure (SPOF)
A SPOF is like a single light switch controlling the power for an entire building: if it fails, everything goes dark. Remove SPOFs with redundant ELBs across AZs, RDS Multi-AZ, Auto Scaling Groups, and CloudFront for content delivery even when origin servers fail.
Proxy types
Forward proxy: sits in front of clients (anonymizes them, controls outbound access). Reverse proxy: sits in front of servers (load balances, hides backend IPs, terminates SSL). NGINX, HAProxy, and CloudFront all act as reverse proxies in common architectures.
Consistent Hashing
Standard hashing remaps everything when you add or remove a server. Consistent hashing maps both data and servers onto a ring, so adding a server only remaps the data “next to” it on the ring. Used by Memcached, Redis, DynamoDB, and Cassandra to distribute data without massive reshuffling. Virtual nodes (vNodes) smooth out uneven distribution.
ACID
Think of a bank vault. Atomicity: either all keys open the vault, or none do. Consistency: the vault is never accidentally left open. Isolation: each person uses a key independently, without interfering with others. Durability: money deposited stays there even if the power goes out.
LRU vs LFU cache eviction
LRU (Least Recently Used): removes the item accessed longest ago. Good for scenarios where recently used items are likely to be reused again. Like clearing old newspapers from a coffee shop table: the oldest untouched one goes first.
LFU (Least Frequently Used): removes the item accessed least often over time. Better when some items are consistently popular. Like a museum keeping exhibits based on how often visitors view them.
SHA hashing
SHA is a family of cryptographic hash functions: one-way, deterministic, collision-resistant. SHA-256 produces a 64-character hex string. Used for data integrity, password storage (with salt), and blockchain. To get an 8-character hash, just truncate: hashlib.sha256(b"hello").hexdigest()[:8]
WebSockets vs HTTP
HTTP is request-response: the client asks, the server answers, the connection closes. WebSockets keep the connection open so either side can send messages at any time, making them ideal for chat, live dashboards, and multiplayer games. After the initial handshake (over HTTP), subsequent messages have minimal overhead.
IP addressing
IPv4 uses 32-bit addresses (~4.3 billion). We ran out. NAT extended the life of IPv4 by letting many devices share one public IP. IPv6 uses 128-bit addresses (340 undecillion), eliminating scarcity. Most networks run dual-stack (both) during the transition.
TCP vs UDP
TCP: connection-oriented, reliable, ordered. Establishes a handshake, retransmits lost packets, guarantees delivery. Use for HTTP, email, file transfer, anything where data integrity matters. UDP: connectionless, fast, no retransmit. Use for video streaming, gaming, DNS, anything where low latency matters more than perfect delivery.
DNS
DNS is the internet’s phone book: turns google.com into 142.250.190.78. Resolution flows from local cache to ISP resolver to authoritative DNS servers. Google’s anycast network routes DNS queries to the nearest server globally, keeping resolution fast and reliable under massive load.
What you’ve built
A mental toolkit for system design: reliability patterns, caching strategies, database tradeoffs, hashing, and communication protocols. These are the building blocks that come up in every senior technical discussion and architecture review, whether you are designing for 10 users or 10 million.
Next steps
- Practice system design with real constraints: given 10M DAU, 1TB storage per year, 99.9% uptime SLA, what does the architecture look like? Quantifying requirements forces clarity.
- Read system design case studies from engineering blogs (Twitter, Netflix, Uber, Airbnb). The patterns repeat across companies.
- Practice articulating tradeoffs out loud: “I chose SQL over NoSQL here because the data is highly relational and ACID guarantees matter for this use case.”
Questions or feedback? Find me on LinkedIn or GitHub.