A scalable system handles increased load without proportionally increasing latency or error rates. There are two primary scaling strategies, and most production systems use both.
Vertical vs Horizontal Scaling
- Vertical (scale up) — add more CPU, RAM, or faster storage to a single server. Simple but has a hard ceiling and single point of failure.
- Horizontal (scale out) — add more servers and distribute load across them. More complex but theoretically unlimited and fault-tolerant.
Load Balancers
A load balancer sits in front of a pool of servers and routes each incoming request to one of them. Common algorithms include round-robin (requests cycle through servers in order) and least-connections (route to the server with fewest active connections).
Caching
Caching stores computed or fetched results in fast memory so subsequent requests skip the expensive operation. Common layers include CDN (static assets at the edge), application-level cache (Redis/Memcached for query results), and database query cache.
Cache Invalidation Strategies
- Cache-aside (lazy loading) — populate cache on miss, read from cache on hit
- Write-through — write to cache and database simultaneously
- TTL (time-to-live) — entries expire automatically after a set duration
- Event-driven invalidation — invalidate cache entries when the underlying data changes