Horizontal scaling

Add more machines behind a load balancer. Capacity grows almost linearly and one failure only removes a fraction of it, but the application must be stateless or share its state elsewhere.

Where it shows up in real systems

  • Application load balancer

    It terminates TLS, routes by path or host and stops sending traffic to an instance that fails its health checks. The algorithm matters: least-connections absorbs uneven requests better than round robin, and sticky sessions create hot spots.

    In production: AWS Application Load Balancer, NGINX, Envoy, HAProxy, Azure Application Gateway

  • Virtual machine

    Throughput grows with vCPUs and depends on the runtime: Go serves several times more requests per core than Python. If memory is below what the runtime needs, the machine swaps and slows down sharply. Several instances behind a load balancer survive the loss of one.

    In production: Amazon EC2, Azure Virtual Machines, Google Compute Engine, OVHcloud Public Cloud

  • Auto-scaling group

    Instances are added when CPU exceeds the target and removed when it falls. New instances need a warm-up period before taking traffic, so sudden spikes still hurt: keep enough minimum capacity. Failed instances are replaced automatically.

    In production: AWS Auto Scaling groups, Azure VM Scale Sets, GCP Managed Instance Groups, Kubernetes HPA

See it break, then fix it

Build an architecture, send it a day of traffic and outages, and learn why it holds or fails. Free to start.

Open the learning path Open the sandbox