Clustering Node.js

Submitted by admin on Saturday, December 24, 2016 - 23:31.
Node.js is single-threaded. As every design decision this one has its benefits and drawbacks. A benefit is - it is significantly harder to "shoot yourself in a foot" with multithreaded code. With languages like C++ or Java, multithreading is a separate subject that is rather advanced and even considered to be a form of a black magic by some developers. So in JavaScript you don't even have to think about scary terms like "deadlock", "race condition" or "memory barrier". What's the downside? Since Node.js is single-threaded, this means that a single Node.js process can utilise one CPU core at most. There's just one thread so there's nothing to run in parallel. If you have a server with 28 cores only one of those cores will be "busy" for any given node.js app. Doesn't sound like a good resource utilisation, does it? Luckily there are many strategies how to scale Node.js horizontally. One of the strategies is by using a core module called "cluster". In this article I'll show how to utilise this module to run multiple Node.js processess and make them "talk" to each other. The basic structure of clustered Node.js app is as follows:
  1. const cluster = require('cluster');
  2.  
  3. if (cluster.isMaster) {
  4.   // This is master node
  5.   const cpuCount = require('os').cpus().length;
  6.  
  7.   for (let i = 0; i < cpuCount; i++) {
  8.     const worker = cluster.fork();
  9.   }
  10.  
  11. } else {
  12.   // This is worker node
  13. }
The processess in cluster can be of two types: master and workers. Master's job is to coordinate workers, spawn them, send messages if required and track if a worker node dies. Let's build a simple cluster-based application that calculates the sum of N random numbers in parallel, using worker processess.
  1. const cluster = require('cluster');
  2.  
  3. if (cluster.isMaster) {
  4.   const cpuCount = require('os').cpus().length;
  5.  
  6.   for (let i = 0; i < cpuCount; i++) {
  7.     const worker = cluster.fork();
  8.     worker.send({ count: 1000000 });
  9.     worker.on('message', (msg) => {
  10.       console.log('Result: ', msg.sum);
  11.     });
  12.   }
  13.  
  14. } else {
  15.   console.log(cluster.worker.id, 'is running');
  16.   process.on('message', (msg) => {
  17.     console.log('Calculating sum of', msg.count, 'numbers');
  18.     let sum = 0;
  19.     for (let i = 0; i < msg.count; i++){
  20.       sum += Math.floor(Math.random()*500);
  21.     }
  22.     process.send({ sum });
  23.   });
  24. }
Once the master process is spawned it sends messages to child processess with the amount ouf numbers that it wants to get a sum of. Worker processess listen to a "message" and get this number. Then, once the job is done, workers send the result back to the master with process.send() function. The communication mechanism used here is IPC - interprocess communication. Using this method is much faster than using network (although, it is limited to processess running on the same host). So now you saw how to distribute work between processess. If you increase the number a little (to make workers run at least 10 seconds) and open the CPU monitor, you'll see that all cores are "hot". Most of the time you start HTTP servers with node, not just calculating numbers. Cluster module has out of the box support for sharing requests between multiple processess. Here's how we can make a clustered HTTP server.
  1. const cluster = require('cluster');
  2. const http = require('http');
  3.  
  4. if (cluster.isMaster) {
  5.   const cpuCount = require('os').cpus().length;
  6.  
  7.   for (let i = 0; i < cpuCount; i++) {
  8.     cluster.fork();
  9.   }
  10.  
  11. } else {
  12.  
  13.   http.createServer((req, res) => {
  14.     res.writeHead(200);
  15.     res.end('Hi, this is node ' + cluster.worker.id + ' of a cluster');
  16.   }).listen(8080, () => {
  17.     console.log('Listening on port 8080');
  18.   });
  19. }
As you see, we are starting as many HTTP servers as we got cores in our computer. For me it starts 8 servers. "But how do they all use the same port" you might ask. This is the feature enabled by cluster module. Worker nodes are sharing the same TCP port and they respond to requests in a round-robin manner (this is a default behavior for all OSes except widows). As you see, it is easy to scale Node.js within one box, cluster module makes it trivial to write code that utilises more than one CPU core at a time. Scaling to multiple boxes is way more interesting, but this is a subject for another article.
Tags
Clustering
node.js

Add new comment

About text formats
  • 94 views