From Nodejs.org

WAT?

• Chrome's JavaScript runtime is called V8, it's one of the reasons Chrome is considered so fast.
• node.js leverages V8's speed on the server!

• Node wants to act as a message passer (more on this later).

• Only one thing gets executed at a time, but libraries allow I/O or other intensive tasks to be executed asynchronously so the script doesn't need to wait for them to finish.

What Makes it

Fast and Scaleable?

• V8 Engine

• Asynchronous libraries

• Event Driven.

Why do we need another server side language/framework?

PHP is slow.

What do you mean by slow?

We actually mean Drupal and we mean it's bad at concurrency.

Why is PHP/Drupal bad at handling concurrency?

• Big memory foot print.
  • If you need to load 100Meg of Code to deliver 4 lines of JSON. . .
• Everything happens in order.
  • Your 100ms database call will hold up the execution of the entire process.
  • Your next 100ms call will do the same, etc.
  • Only after all the database calls have completed can we return the 4 lines of JSON.

In node:

huh?

Imagine a King with servants

• Every morning the servants line up.
• One at a time they come into his throne room.
• They report on what they've done.
• Sometimes the king gives them more to do.
• Always one at a time, so the king can focus.

Code Please.

  var fs  = require('fs');
  var sys = require('sys');

  fs.readFile('treasure-chamber-report.txt', function readFile(report) {
    sys.puts("oh, look at all my money: "+report);
  });

  fs.writeFile('letter-to-princess.txt', '...', function writeFile() {
    sys.puts("can't wait to hear back from her!");
  });

Explain the Code Please.

• Code gives node 2 things to do.
• Each callback gets fired one at a time
• Until one callback is fired all others wait in line

However

• We don't need to wait for the file being read, to write the file.
• Both System IO calls are started independently.

Don't need to wait for one to finish before the next starts!

Why not write non-blocking/event-driven PHP

There are event driven Async PHP frameworks.

For example: PRADOTM

Libraries.

• PHP libraries are primarily blocking and not event driven.
• Node.js libraries are almost all non-blocking and event driven.

What is node.js good for

• Rapid prototyping
• Bursty load
• API glue
• Queue-ish stuff
• "Real-time" applications
  • Easy example: chat
• Message passing!

What is node.js NOT good for

• General purpose web server
• (Large) static assets
• Complex systems (business logic)
• Lots of processing - use C or PHP

What We've used it for

• RSS Hub
• Interactive game
• Realtime video commenting
• IRC Bot
• Parsing HTML

What about the community?

• Welcoming community
• Package manager that kicks butt
  • Over 16k packages
• Very fast moving project
  • 63 Releases in the last year and half
  • BUT has really good test coverage
• Lots of Frameworks
• Drupal module

One more thing . . .

Share client and server side code.

Connecting to the server

• SSH into the server from your local machine

  $ ssh YOU@nodejs.4kclass.com
  

• Pro-tip: You may want to keep two terminal sessions open to the server so you can edit files while running a node process or edit two files at once in different windows.

Editing files

• Use your preferred editor on server (we like vim but won't judge if you use emacs -- but we can't help you either).

• If you want to edit files locally, SFTP with the credentials you were provided

  $ sftp YOU@nodejs.4kclass.com
  

• You can also use your preferred SFTP client.

File structure

• All course code is located in

  ~/nodejs

• Familiarize yourself with the files and directories here, you'll be spending most of your time in the following directories:

  ~/nodejs/js101
  ~/nodejs/exercises/drupal
  ~/nodejs/exercises/server
  

Commands you'll be using

• cp [src] [dst] - to copy a file
• mv [src] [dst] - to move or rename a file
• rm [file] - to delete a file
• node [file] - to start a node process (or just node to use the node REPL shell).
• CTRL+C - to stop a node process that does not automatically exit.

Port Number

• A unique port has been assigned to each of you, it's in the ~/port-xxxxx file in your home directory. You'll need this when we start a node server.
• The ~/config.json file in your home directory can be used to inject settings in your node code (more on that later).

Your first Server.

~/nodejs/hello/app.js

• Respond to the HTTP request with a hello world message

  var http = require('http');
  http.createServer(function hello(request, response) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
  }).listen(1337);
  console.log('Server running at http://127.0.0.1:1337/');

change 1337 to your port number.

Express

• A web application framework for node.js
• Provides:
  • Static asset handling
  • Dynamic router
  • Connect middleware

Getting started

• Use the express command to set up some scaffolding for a new application:

  $ cd ~/nodejs/exercies
  $ express myserver
  $ cd myserver
  $ npm install

Copy your personal Config file.

• We're going to make a few changes to support the application we're going to build:
• Copy the config.js file and client directory from your home directory to the example dirctory:

  $ cp ~/config.js ~/nodejs/exercises/myserver/config.js
  

Edit app.js

Include the config we copied by adding it after the where path is required. Don't forget to remove the ";" after require('path').

-  , path = require('path');
+  , path = require('path')
+  , config = require('./config');

Use the port number from the config file.

-  app.set('port', process.env.PORT || 3000);
+  app.set('port', config.port);

Try it.

What we'll build.

• We'll ping the node server with with node title
• We'll create a list of Drupal nodes in our app.

Create the Node.js Endpoint.

  app.post('/ping', function postPing(request, response){
    console.log(request.body);      // your JSON
    response.send(request.body);    // echo the result back
  });

Create a Drupal module to send the title to node.

Example Drupal Module code.

  $data = array('title' => $node->title);
  $data = drupal_json_encode($data);
  $uri = variable_get('ping_nodejs_uri','http://localhost:3000/ping');

  $options = array();
  $options['headers'] = array('Content-Type' => 'application/json');
  $options['method'] = 'POST';
  $options['data'] = $data;
  $response = drupal_http_request($uri, $options);

  if ($response->code != 200) {
    drupal_set_message("Got an error: " . $response->error, 'error');
  }
  else {
    drupal_set_message('Successfully contacted node server. Response was: ' . $response->data);
  }

How to stop Code-blocking

async/

Remember the King with servants?

• Every morning the servants line up.
• One at a time they come into his throne room.
• They report on what they've done.
• Sometimes the king gives them more to do.
• Always one at a time, so the king can focus.

Syncrounous example

Checkout the order of the log messages.

  console.log("I'm going to read a file synchronously.");

  syncContents = fs.readFileSync(__filename);

  console.log("I'm done reading the file synchronously.");
  console.log('I read ' + syncContents.length + ' characters.');
  console.log("Now I'm ready to do the next thing.");

Asynchronous Example

Now see how the log messages have changed.

  console.log("I'm going to read a file asynchronously.");
  fs.readFile(__filename, function readFile(err, data) {
    asyncContents = data;
    console.log("I'm done reading the file asynchronously.");
    console.log('I read ' + asyncContents.length + ' characters.');
  });
  console.log("Now I'm ready to do the next thing.");

Results

Why have both?

Why have both?

• Sometimes simplifies code flow.
• Useful if you need to stop code from executing until the operation is completed.
• Generally you should ask yourself "Should this be a callback instead."

One last note.

Don't do complicated stuff in the event looop.

  var start = Date.now();

  setTimeout(function timeout1() {
    console.log('We started execution ' + (Date.now() - start) + 'ms ago.');

    for (var x = 0; x < 3999999999; x++) {}

    console.log('Done with timeout1().');
  }, 1000);

  setTimeout(function timeout2() {
    console.log('We started execution ' + (Date.now() - start) + 'ms ago.');
    console.log('Done with timeout2().');
  }, 2000);

Keep it simple.

Don't put complicated logic in the main event thread.

• Long sorts/searches.
• Long Math (fibbonacci sequence).
• Processing lots of raw data.

How to get around this problem:

• Fork process
• Hand off to other program and listen for event.
• Write C
• Use a queue

require()

• Used to include built in libraries, libraries from package manager and local files.

• Automatically handles dependenies.

  ~/nodejs/require/modules

module_a.js

console.log('this is a');

module_b.js

console.log(' this is b');

main.js

  // Note the relative paths, leaving this off for libraries you
  // define is a common mistake.
  require('./module_a');
  require('./module_b')

$ node main.js

Dependency Handling AKA Package.json

  {
    "author": "Four Kitchens  (http://fourkitchens.com/)"@fourkitchens.com>,
    "name": "npmExample",
    "description": "An example from the world famous Four Kitchens Node.js training.",
    "version": "0.0.1",
    "homepage": "http://fourkitchens.com",
    "repository": {
      "type": "git",
      "url": "git://github.com/fourkitchens/train-node.git"
    },
    "engines": {
      "node": "~0.8.8"
    },
    "dependencies": {},
    "devDependencies": {},
    "optionalDependencies": {}
  }
@fourkitchens.com>

package.json reference

NPM

• Node's Personal Manservant
• Collection of over 16k community built libraries
• Handles installation and dependencies
• Everything from GPIO libraries to a few CMSs

NPM Commands

• npm install - install module
  • add -g to install at the system level.
• npm install --save - installs module and adds it to your local package.json file.
• npm install - installs everything from the local package.json file.
• npm init - creates an empty package.json file
• npm search - search for modules

NPM help (if --help is too old school for you.)

Try it

  $ mkdir ~/nodejs/require/colorsExample
  $ cd ~/nodejs/require/colorsExample
  $ npm init
  $ npm install colors

To add a library to your package.json file at install time add --save:

  npm install --save colors

• Create a file called colorsExample.js in the colorsExample directory
• Here's an example of using the colors library, give it a try and run the script with node colorsExample when you're done editing.

  // colorsExample.js
  var colors = require('colors');

  console.log('hello'.green); // outputs green text

More fun with colors

  console.log('i like cake and pies'.underline.red) // outputs red underlined text
  console.log('inverse the color'.inverse); // inverses the color
  console.log('OMG Rainbows!'.rainbow); // rainbow (ignores spaces)

Where does require() look for files?

• Core libraries
• modules in the node_modules directory
• locally defined modules if the module is prefixed with a path
  • Paths can be absolute, but this is highly discouraged, use the relative paths or package your own module.

the mystery explained

Getting a list of Drupal Nodes.

Install and Configure Services module.

Create & theme a route to show a list of nodes.

Add a new library

npm install --save superagent

Create the Express middleware AKA route.

Create file: */routes/nodes.js

  var request = require('superagent');

  /**
   * Get Nodes function
   *
   */
  module.exports  = function nodes (url, fn) {
    request.get(url)
      .end(function gotNodes(res) {
        if (res.body) {
          return fn(null, res.body);
        }
      });
  };

Change to our theme engine.

handlebars, hbs handlebars for Expresss

npm install --save hbs

add the themeing engine to our app.

-  app.set('view engine', 'jade');
+  app.set('view engine', 'html');
+  app.engine('html', require('hbs').__express);

Create the layout.

Renders the page.

• Create the file /views/layout.html
• gist with layout

Create the template

Render's the content.

Create the file /views/nodes.html

gist with template

Add code to the app.

  app.get('/nodes', function getNodes(req, res, next) {
    nodes('http://instructor.nodejs.4kclass.com/exercises/drupal/rest/node.json', function renderNodes(err, nodes) {
      console.log(nodes);
      if (err) return next( err);
      res.render('nodes', { results: nodes });
    });
  });

What is Socket.io

Server

  var io = require('socket.io').listen(80);

  io.sockets.on('connection', function onConnection(socket) {
    socket.emit('news', { hello: 'world' });
    socket.on('my other event', function onMyOtherEvent(data) {
      console.log(data);
    });
  });

Client

  var socket = io.connect('http://localhost');
  socket.on('news', function onNews(data) {
    console.log(data);
    socket.emit('my other event', { my: 'data' });
  });

What you'll build

• A backend to this...

What you'll build

• A loop to fill out content in the stories region
• A socket.io server. The client is currently aware of three entities and two events each...

Story

• story - contains a title, description, and link.

  {
    title: 'Story title',
    description: 'Story body text',
    link: 'URL to the story'
  }
  

• socket.io events:
  • newStories - array of story objects to add to the region.
  • refreshStories - array of story objects to replace what's in the region with.

Message

• message - contains a name, message, and time.

  {
    name: 'Message Author',
    message: 'Message body text!',
    time: new Date()
  }
  

• socket.io events:
  • newMessages - array of message objects to add to the region.
  • refreshMessages - array of message objects to replace what's in the region with.

Image (bonus)

• image - contains a picture (URL) and link.

  {
    picture: 'URL to the image',
    link: 'URL to where the image should be linked to'
  }
  

• socket.io events:
  • newImages - array of image objects to add to the region.
  • refreshImages - array of image objects to replace what's in the region with.

Bonus challenges!

Bonus challenges, cont

• Hint: see the next set of slides for some tips on working with databases in node.

Where's the code?

  ~/nodejs/exercises/server/app.js

• This is where you'll be living for the remainder of the class. The code is highly commented.
• Anything marked with TODO is a task we're going to guide the class through.
• Anything with (bonus) is a bonus task that we'll guide you through if we get to it, but feel free to tackle these on your own if you complete the other tasks.

Create some story content.

  // Keep a counter.
  var count = 0;
  /**
   * Notifies the client of new stories.
   */
  setInterval(function newStories() {
    // Increment the counter.
    count++;

    // Create some story content to send to the client.
    var newStories = [
      {
        title: 'New story ' + count,
        description: 'New story text for ' + count,
        link: 'http://fourkitchens.com'
      }
    ];

    // Broadcast the new stories to all clients.
    io.sockets.emit('newStories', newStories);
  }, config.pollInterval || 10000);

Create stories from Drupal

  /**
   * Notifies the client of new stories.
   */
  setInterval(function newStories() {
    nodes('http://instructor.nodejs.4kclass.com/rest/node.json', function gotNodes(err, nodes) {
      if (err) { return next(err); }

      var newStories = [];

      nodes.forEach(function eachNode(node) {
        newStories.push({
          title: node.title,
          description: node.title + ' (nid: ' + node.nid + ')',
          link: 'http://instructor.nodejs.4kclass.com/node/' + node.nid
        });
      });

      io.sockets.emit('newStories', newStories);
    });
  });

Create stories from Drupal

• Create new content on your Drupal site, you should see it show up on the client the next time the site is polled.
• Want to make it better? Keep a variable with the nodes you've already received and only emit the new ones.

Welcome new clients

  app.get('/messages', function getMessages(req, res) {
    var messages = [
      {
        name: 'Training bot',
        message: 'Welcome to node.js training!',
        time: new Date()
      }
    ];

    res.json(messages);
  });

Message chat

  app.post('/message', function postMessage(req, res) {
    res.writeHead(204);
    res.end();

    var newMessages = [req.body];
    io.sockets.emit('newMessages', newMessages);
  });

Message chat

• Try opening your client site in another browser and start sending messages!
• Want to make it better? Store the new messages so you can serve chat history to new clients.

Overview

• As its name implies, streams are a continuous flow of information that can be diverted or processed as it comes in.

Overview

• As its name implies, streams are a continuous flow of information that can be diverted or processed as it comes in.
• This is incredibly efficient, because you don't have to wait until a stream is finished before working with any data that's come in.

Overview

• As its name implies, streams are a continuous flow of information that can be diverted or processed as it comes in.
• This is incredibly efficient, because you don't have to wait until a stream is finished before working with any data that's come in.
• Fundumantal abstraction in node.js I/O.

Overview

• As its name implies, streams are a continuous flow of information that can be diverted or processed as it comes in.
• This is incredibly efficient, because you don't have to wait until a stream is finished before working with any data that's come in.
• Fundumantal abstraction in node.js I/O.
• Can be readable (reading a file), writeable (HTTP response) or both TCP connection.

Best part is. . .

Best part is. . .

. . . You are already using them.

First code we wrote used a stream.

  var http = require('http');
  http.createServer(function hello(request, response) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
  }).listen(1337);
  console.log('Server running at http://127.0.0.1:1337/');

Lets give it a shot.

~/nodejs/streams

• Read a file from the file system and write it out to the console.

On the stream data event write to console.log.

  file.on('data', function(chunk){
    console.log(chunk.toString());
  });

For comparison lets use readFile instead

fs.readFile('index.html', function(err, data) {
  if (err) throw err;
  console.log(data.toString());
});

For comparison lets use readFile instead

fs.readFile(filename, [options], callback)

• Reads the contents of the file asyncrounsly.
• fs.readFile Docs

console.log the file contents using readFile().

Now create an HTTP server using readFile

  fs.readFile('index.html', function (err, data) {
    if (err) {
      res.statusCode = 500;
      res.end(String(err));
    }
    else res.end(data);
  });

Pipe

• Method on streams that allows you to route the data from one stream to another directly
• Allows for some very powerful and performat opperations.
• works like the unix pipe '|'

history | grep node

Server using Pipe.

  var file = fs.createReadStream('index.html');
  file.pipe(res);

Adding a stream of images from flickr to our little webapp.

• Helper code to get a feed from flickr and pipe into into a stream. ** https://gist.github.com/mirzu/5828095

• New emitter for images. https://gist.github.com/mirzu/5828092

Overview

• One of the easiest places to see asynchronous programming:
• Application makes call to database and immediately returns to the event loop.
• Later, database calls back with results.
• For the more advanced examples in this class you'll be given the opportunity to work with a database, either MySQL or MongoDB.

MySQL

• http://www.mysql.com/
• Relational database system
• If you're a Drupal developer you're probably pretty familiar with it, but the libraries we'll be using to interact with it aren't as abstracted as DBTNG.
  • i.e. you'll have to write actual SQL queries ಠ_ಠ

Node.js and MySQL

• We'll be using the node-mysql library.
• $ npm install mysql@2.0.0-alpha3

  var mysql = require('mysql');
  var conn = mysql.createConnection({
    host: 'localhost',
    user: 'you',
    password: 'unsafe'
  });
  conn.connect();

  conn.query('SELECT "Hello world" AS message', function selectResult(err, rows, fields) {
    if (err) {
      throw err;
    }

    console.log(rows[0].message);
  });
  conn.end();

MongoDB

• http://www.mongodb.org/
• Schemaless document store
• Documents are stored as BSON which is very similar to JSON and therefore extremely convenient to work with in node.js!

MongoDB Examples

• There's not a real query language in the same sense as MySQL and since the database is schemaless you can throw whatever you like at it:

  $ mongo

  db.test.insert({
    title: 'test',
    message: 'Hello world'
  });
  db.test.find({ title: 'test' }).pretty();
  db.test.update(
    { title: 'test' },
    { $set: { message: 'Yo dawg' } }
  );
  db.test.find({ title: 'test' }).pretty();

Sub documents

• One of the most useful parts of MongoDB is the ability to store "sub documents" or objects within a primary document:

  db.test.update(
    { title: 'test'},
    { $set: {
      speaker: {
        name: 'Elliott Foster',
        occupation: 'Hipster Technologist',
        interests: [
          'JavaScript',
          'beer',
          'bicycles'
        ]
      }
    } }
  );
  db.test.find({ 'speaker.name': 'Elliott Foster' }).pretty();

Node.js and MongoDB

• We'll be using the mongode library to interact with MongoDB

  var mongode = require('mongode');
  var test = mongode.connect('mongo://127.0.0.1/test');
  var collection = test.collection('test');
  var doc = { title: 'test2', message: 'node and mongo, yay!' };
  collection.insert(doc, {safe:true}, function insertResult(err, objects) {
    if (err) {
      console.error(err.message);
      process.exit(1);
    }
    collection.findOne({ title: 'test2' }, function findOneResult(err, document) {
      if (err) {
        console.error(err.message);
        process.exit(1);
      }
      console.log(document.message);
      process.exit(0);
    });
  });

Roll your own

  // basic imports
  var events = require('events');
  var util = require('util');

  function Poll(feed) { this.feed = feed; };
  util.inherits(Poll, events.EventEmitter);

  Poll.prototype.getStories = function(callback) {
    var self = this;

    /**
     * Assume fetchStories grabs an array of content and
     * executes this callback on completion.
     */
    fetchStories(self.feed, function gotStories(err, newStories) {
      if (!err && newStories.length) {
        self.emit('fed', null, newStories);
      }
    });
  };
  exports.Poll = Poll;

Hashes (AKA Objects)

• A hash is an object, they can contain properties:

  var awesomeCompany = {
    name: 'Four Kitchens',
    location: 'Austin, TX'
  };

  console.log(awesomeCompany.name + ' is awesome!');

Hashes (AKA Objects)

• They can also contain functions:

  var awesomeCompany = {
    name: 'Four Kitchens',
    location: 'Austin, TX',
    knowsNode: function() {
      return true;
    }
  };

  if (awesomeCompany.knowsNode()) {
    console.log(awesomeCompany.name + ' knows node!');
  }