Horizontal Scaling (Scaling Out)
Adding more machines to a system to handle increased load.
Detailed Explanation
Horizontal scaling is the preferred method for modern cloud applications. Instead of buying a more powerful server (Vertical Scaling), you add more identical servers to a cluster. This approach offers better fault tolerance and near-infinite scalability, as you are not limited by the hardware ceiling of a single machine.
Quick Summary
Horizontal scaling adds more identical machines (or pods) and distributes work across them with a load balancer. It removes the hardware ceiling of a single box and provides redundancy as a side effect.
Key Takeaways
- Works only when the workload is stateless or state is offloaded to a shared store (database, cache, object storage).
- Pairs naturally with autoscalers (Kubernetes HPA, AWS ASG) that add/remove instances based on metrics.
- Cost scales roughly linearly with capacity, no "big iron" premium like vertical scaling at the top end.
- Fault tolerance is a free byproduct: lose one node, the others absorb the load.
- Coordination overhead (consensus, distributed transactions) grows with the cluster, not every workload benefits.
When to use it
- Web tiers and API servers serving stateless HTTP requests.
- Worker queues processing background jobs in parallel.
- Read-heavy databases with replicas distributing query load.
- Cloud-native apps designed for elastic, demand-based scaling.
Common Mistakes
- Storing session or upload state on local disk; the next request hits a different machine and breaks.
- Adding more instances when the bottleneck is the database, the new instances just queue against it.
- Skipping graceful shutdown, so scale-in events drop in-flight requests.
- Underestimating the cost of cross-instance coordination (locks, sticky sessions) until it dominates latency.
Horizontal Scaling (Scaling Out), Frequently Asked
Horizontal vs. vertical scaling, which should I do first?
Vertical scaling is simpler, bigger box, no code changes, and often enough for a long time. Move to horizontal when you hit hardware limits, need redundancy, or face spiky load that benefits from elastic capacity.
Can I horizontally scale a relational database?
Read replicas yes; writes are harder. Solutions exist (sharding, Vitess, CockroachDB, Spanner), but they trade complexity and constraints. Most teams scale the app horizontally and the database vertically until very late.
What workloads do not scale horizontally?
Anything with strong serializable consistency requirements, heavy cross-instance coordination, or shared in-memory state. Game servers, ledgers, and certain ML training jobs often need careful design or vertical scaling instead.