Load Balancing Strategies: Managing Traffic

← Back to Architecture Hub

Load Balancing Strategies: Managing Traffic

1. Round Robin: The "Simple Fair" Way

The Logic: Request 1 -> Server A. Request 2 -> Server B. Request 3 -> Server C.
The Problem: What if Server C is a 10-year-old machine and Server A is a brand new supercomputer? Round Robin will crush the weak server.
The Fix: Weighted Round Robin. You tell the load balancer: "Send 80% of traffic to Server A and only 20% to Server C."

2. Least Connections: The "Intelligent" Way

The Load Balancer keeps track of how many active users are on each server.

"Server A has 100 users. Server B has only 2."
It sends the next user to Server B. This is the best strategy for "Long-lived" connections (like a Socket.io chat or a Video Stream) where one user might stay on the server for an hour.

3. Layer 4 vs. Layer 7 Balancing

Layer 4 (Transport): Faster. It only looks at the IP address and Port. It doesn't "Read" the data inside the message. (Good for simple load balancing).
Layer 7 (Application): Smarter. It can "Read" the HTTP headers. "If the user is logged in, send them to the 'Pro Server'. If they are a guest, send them to the 'Ad Server'." In 2026, Layer 7 is the standard for complex Microservice environments.

4. Consistent Hashing: The Caching Secret

If you use simple "IP Hash," a user always goes to the same server.

The Problem: If you add a new server, the hash math changes, and EVERY user gets moved to a different server. This crashes your "Layer 1 Cache" (Module 183).
The Solution: Consistent Hashing. It ensures that when you add or remove a server, only $1/N$ of the users have to move. It is the gold standard for high-performance databases and CDNs.

Frequently Asked Questions

Can NGINX be a Load Balancer? YES. NGINX is the world's most popular software load balancer. It can handle $100,000$ connections per second on a single $16$-core box. For most startups, an NGINX Load Balancer is all you will ever need.

What is 'Health Checking'? The Load Balancer sends a "Heartbeat" (a ping) every 5 seconds. If a server doesn't answer, it's marked as "DEAD" and the traffic stops. This is the secret to building a system that "Heals itself"—the user never sees the error because the traffic moved before they clicked.

Key Takeaway

Load balancing is the "Architecture of Reliability." By mastering the distinction between Transport and Application layers and the precision of Least Connections routing, you gain the ability to build massive server clusters that are invisible to the user. You graduate from "Managing a computer" to "Orchestrating a Fleet."

Part of the Software Architecture Hub — engineering the traffic.