Introduction

Recently, I organised a “Bits & Bites” session, introducing my colleagues to Node.js, Express and MongoDB. At SynTouch, we regularly have knowledge sharing sessions and workshops. Usually they are held late afternoon at a central location and include dinner, hence the name “Bits & Bites”! The reason I have chosen this specific subject has to do both with past as well as current interests. Some time ago, for a proof of concept, a number of microservices were required for demonstration purposes and to implement some required features for my customer which at that time were missing from the product (and coming to think of it, a year later they are still missing … probably on the road map). This proof of concept has been described in previous blog posts, (click here and here). Having visited the Serverless Conference in Paris (click here), I also want to organize an additional Bits & Bites on this interesting topic. However, following a few discussions, we feel the need to introduce you to Node.js first.

So … What is Node.js?

Node.js is a scripting language, building on the language we now know as JavaScript (an ECMAScript implementation). This language was originally created by Netscape and implemented in their browser for making web pages more dynamic. Nevertheless, Ryan Dahl created a server side implementation for this JavaScript engine, based on the same engine that the Google Chrome browser shares with the open source Chromium Browser, i.e. the Google v8 JavaScript engine. This engine is one of the fastest in its kinds, with the advent of the engine to the server and the additional of a whole set of server side libraries, JavaScript escaped it’s existence as a client-side only language and got the future more exciting than only dynamically showing widgets and responding to user click events. From this point onwards, it was possible to write server side applications in JavaScript that could access the file system, perform queries on databases and execute all kinds of other network I/O, just like server side programs written in Python, Ruby or Java. The core modules of Node.js are written in C++, but most of the standard library is also written as JavaScript code:

What are Node.js’ strengths?

Node.js has become quite popular as a development platform since its inception in 2009, as reflected in the 2018 StackOverflow Developer Survey, where both JavaScript and Node.js ranks as the most popular technologies for the group of professional developers:

But what could be the reason for Node’s popularity? Well, one of its unique features is the fact that all I/O will be performed in an evented, non-blocking way. This is relevant because I/O, the process of moving data into the CPU for processing, is notably slow. As Ryan recalled during the unveiling of Node, especially disk and network I/O are orders of magnitude slower than memory (L1/L2 cache or RAM) access:

In traditional programming environments, I/O was performed in a blocking mode: the program requested data and issued an API call, retrieving data from the file system, from a database or over the network as another API Call. The requesting program would block until the data was received back from its API call and usually the thread was put to sleep. To serve more than just a single client, new threads would be spun up to handle additional requests – this is a very costly practice, as all of the threads waiting for the I/O to complete are still consuming resources. Needless to say, this solution does not scale well under load.

Using a simple example, supposedly fetching two customers in sequence from a database, I showed how Node.js implements the non-blocking I/O by taking a callback function. This callback function is invoked as soon as the operation completes, but in the meantime, Node will just continue with the next statement, whereas the synchronous example will block for the first call and wait for the results:

Blocking (synchronous) implementation

var sync = require('./lib/doSyncStuff');

function p(data) {
    console.log(JSON.stringify(data));
}

// retrieve two customers from DB and show
p(sync.doSyncStuff('Brouwerij Het Uiltje'));
p(sync.doSyncStuff('Raboboerenleenbank'));

console.log('Today is ' + new Date());

Non-blocking (asynchronous) implementation

var async = require('./lib/doAsyncStuff');

function p(data) {
    console.log(JSON.stringify(data));
}

// retrieve two customers from DB and show
async.doAsyncStuff('Janssen & Tilanus', p);
async.doAsyncStuff('Schmidt Zeevis', p);

console.log('Today is ' + new Date());

The following slide from the presentation shows what the difference looks like when plotting the executing code in time elapsed:

As demonstrated in the diagram above, in the blocking I/O model all tasks will execute sequentially, whereas the non-blocking I/O model will allow new tasks to execute during the time in which external I/O is awaited. For web applications and web APIs under load, this can be a significant improvement in both efficiency and response time. As a real-life example comparing blocking versus non-blocking I/O, consider two web servers implemented around these models, Apache and Nginx, and their performance in the C10K challenge:

As illustrated in the diagrams above, the non-blocking I/O models is clearly superior in terms of concurrency and memory usage. I compared this behaviour of collapsing under load to the White Rabbit of Alice in Wonderland:

Huge Package Ecosystem

Another significant plus of Node.js is that it has a HUGE ecosystem in free third party modules in the NPMJS repository. Currently, NPM boasts itself as the largest open source ecosystem:

The current count of diferent packages for npm is over 677,000! The download numbers speak for themselves … The big advantage for me as a developer is not to implement certain base functionality, but being able to rely on popular, battle-tested third party modules! This let’s me focus on the actual logic that I need to implement instead of the umpteenth solution to some header parsing challenge.

The Express Framework

In the last part of my introduction, I touched upon the use of Express Framework for creating web APIs (not being a front-end developer, I only mentioned its use for building Web Applications) by showing how it extends Node’s basic HTTP-server by layering standard and custom middlewares and request handlers:

A simple demo showed off how to create a WEB API with just a few lines of actually code … creating an alternative “Hello World” using the ‘reneFrogger’ callback, that responds to two different endpoints:

Start your (V8) Engines!

As I like to learn by exploring some code, I thought this was a nice and gentle introduction to Node.js and decided that the participants could start their Virtual Box images to explore and experiment with Node.js themselves. For the practical part, I took the environmental-aware approach and recycled some of the materials from a Node.js workshop I attended myself, plus some additional exercises:

– building a simple Node.js CLI tool to count: lines, words and characters for one or more files

– building the ultimate BeerAPI, using Node.js, Express and MongoDB!

Samples and material can be found on GitHub, enjoy!