by Neil Lobo, Software Engineer

Part of the popularity of Node.js is the event loop. Rather than take the multithreaded approach of traditional frameworks, Node.js uses an event-driven, non-blocking I/O model. This  gives it a lighter memory footprint, is more lightweight and efficient, and scales well for data-intensive real-time applications.

This event driven non blocking approach works well for applications where the bottleneck is I/O or network activity. In contrast, if the application require CPU intensive tasks, then Node.js is probably a poor choice.

However, working with bare Node.js requires use of asynchronous callbacks which lead to the infamous callback hell. This leads to logic that is hard to follow

//Callback Soup/Spiral Callback of Doom
function doAsync1(function () {
    doAsync2(function () {
      doAsync3(function () {
        doAsync4(function () {
          //finally do something });
        });
      });
  });

Error handling and nested callbacks are uncomfortable to write, and their existence makes code difficult to maintain and scale. There are some techniques to overcome these issues, the most popular being Fibers and Promises.  I want to discuss the former as Meteor uses Fibers to achieve synchronous like code over the event loop.

Fibers

For those not familiar with it, Fibers is a Computer Science construct which use a model of cooperative multitasking (unlike threads use pre-emptive multitasking)

Threaded code may be interrupted at any point, even in the middle of evaluating an expression, to give CPU cycles to code running in another thread. With fibers, these interruptions and context switches are not determined by the CPU or a lower level process; they are determined by the programmer who decides where his code is going to yield and give CPU cycles to other fibers.

Here’s an example of typical Node.js code that uses a callbackFunction that is passed the result.

getDataFromDisk = function(fileName, key, callbackFunction) { 

  var result; 

  fs.readFile('/path/fileName', function(err, res) {
    if (err) console.log(err); else {
    result = transform(res, key);
    callbackFunction(result);
  }}};

Lets aim to write this is in a more synchronous style

getDataFromDisk = function(fileName, key) {
  var result;
  fs.readFile('/path/fileName', function(err, res) {
    if (err) console.log(err);    else result = transform(res, key); 

  }
  return result;// will always be undefined
};
// We try to use getDataFromDisk return the value, then
// print it out, all synchronously.
var result = getDataFromDisk('helloWorld',key);
console.log(result);
// undefined

The above code will always return undefined, as the event loop will move on to the return result line (*) without waiting for the result of the callback.

Using Fibers

Fibers are a class of container functions that can be used to block a sub-routine while waiting on some I/O or network activity without blocking the entire process.

var Fiber = Npm.require('fibers');// getDataFromDisk function using Fibers
getDataFromDisk = function(filename, key) {var fiber = Fiber.current; //get the current Fiber
fs.readFile('/path/fileName', function(err, res) {
if (err) console.log(err); else{
/* Resume execution of this fiber. What’s passed to fiber.run will become the value returned by Fiber.yield below */
fiber.run( transform(res, key) );
var result = Fiber.yield();
return result;};
// Finally we wrap our code in a Fiber, then run it
Fiber(function() { 

var result = getDataFromDisk('helloWorld', key);
console.log(result);
}).run();

Presto, we have synchronous looking code that is still being run asynchronously on the event loop.

For more details about Fibers check out (https://github.com/laverdet/node-fibers)