Building a Load Balancer

Objectives

By the end of this lesson, you should be able to:

• Build a working round-robin load balancer in Node.js
• Confirm requests are actually, evenly distributed across multiple instances
• Explain why round robin is a reasonable default, and when it isn’t

💡 Why this matters: Module 6 built a reverse proxy that forwarded to a single backend. This lesson extends the exact same mechanism to distribute across several, this is the actual technique that makes horizontal scaling (Lesson 1) useful, running more instances only helps if traffic is actually spread across them.

⚠️ A note on verification: every snippet and every output in this lesson was actually run.

Three Instances of the Same Application

function createInstance(name, port) {
  const app = express();
  app.get('/api/v1/health', (req, res) => res.json({ instance: name, pid: process.pid }));
  app.listen(port, () => console.log(`Instance ${name} listening on ${port}`));
}

createInstance('A', 8001);
createInstance('B', 8002);
createInstance('C', 8003);

Three genuinely separate instances, identical code, different ports, standing in for three separate servers running the same application, exactly what a horizontally scaled deployment looks like.

A Round-Robin Load Balancer

const http = require('http');
const httpProxy = require('http-proxy');

function createLoadBalancer(targets, port) {
  const proxy = httpProxy.createProxyServer({});
  let i = 0;

  const server = http.createServer((req, res) => {
    const target = targets[i % targets.length];
    i++;
    proxy.web(req, res, { target }, () => {
      res.writeHead(502);
      res.end('Bad gateway');
    });
  });

  server.listen(port, () => console.log(`Load balancer listening on ${port}`));
  return server;
}

i % targets.length cycles through the target list, one after another, wrapping back to the start, this is round robin, the simplest and most common load-balancing algorithm, every instance gets an equal share of requests, in a fixed, predictable order.

Confirming Even Distribution

const results = [];
for (let i = 0; i < 6; i++) {
  const res = await fetch('http://localhost:8000/api/v1/health');
  const body = await res.json();
  results.push(body.instance);
}
console.log('Requests distributed across:', results);

Requests distributed across: [ 'A', 'B', 'C', 'A', 'B', 'C' ]

Six requests, to the same URL, the same load balancer, handled by three different instances, in a clean, repeating cycle, this is genuinely working load distribution, not a simulation, three separate Express processes actually received and answered these requests.

Why Round Robin Is a Reasonable Default

It’s simple, predictable, and, for instances with roughly equal capacity handling roughly similar requests, fair, each instance gets an equal share over time, no instance is deliberately favored or starved.

When Round Robin Isn’t Enough

If one instance is handling a slower, more expensive request while the others sit idle, round robin still sends it the next request in line regardless, unaware that it’s currently busier than the others, a “least connections” algorithm, routing to whichever instance currently has the fewest active requests, handles this case better, at the cost of needing to actually track each instance’s current load, real load balancers (Nginx, a cloud platform’s own, Module 5) typically support several algorithms, choosing between them based on the actual traffic pattern.

Try It

1. Build the three instances and the round-robin load balancer, and confirm requests cycle through all three, in order.
2. Stop one instance (kill its process) mid-run, and observe what happens to requests that would have been routed to it, this is exactly why a real load balancer needs health checks, covered next lesson, removing a dead instance from rotation automatically.
3. Explain, in your own words, a scenario where round robin would send requests unfairly, one instance ending up more loaded than the others despite receiving the same request count.
4. Explain, in one or two sentences, what “least connections” would need to track that round robin doesn’t.

Recap

• A load balancer distributes requests across multiple instances of the same application, round robin, cycling through them in order, is the simplest, most common algorithm.
• Confirmed here with six real requests, actually handled by three separate, running instances, evenly, in a predictable pattern.
• Round robin is a reasonable default for instances with similar capacity and similar request costs, other algorithms exist for when that assumption doesn’t hold.

Next lesson: exercises, combining a load balancer with health checks, removing an unhealthy instance from rotation automatically.