Node.js and Redis

Using Redis to scale Caches, Events, and Queues up and accross Node.js Applications

Ryan-Sandy Lee

Web     ryansandy.com Github ryan-sandy

Overview

• Definitions
• Introduction
• Caches
• Event Emitter
• Queue

Definitions

• Scaling Up
  • Adding additional instances (copies) of the same application
• Scaling Out
  • Adding functionality to different code bases

Introduction

• Redis
  • No-Sql
  • In-memory data structure store
  • Strings, hashes, lists, sets, sorted set, bitmaps, hyperlogs, ...
  • redis.io
• Node.js
  • JavaScript interpreter on the server
  • www.nodejs.org

Introduction

• Redis Commands
  • set, get, hset, ...
    • redis.io/commands
  • redis-cli
  • redis-commander
• Node.js Libraries
  • ioredis
  • node_redis
    • hiredis

Simple Example

var redisDriver = require('redis');
var redis = redisDriver.createClient();

redis.set("forever", "forever-value", function (err) {
  redis.get("forever", function (err, val) {
    console.log(val);
    redis.end();
  });
});

forever-value

Cache

• Design Pattern
  • Store data with an expiration
  • Retrieve quickly
• Commands
  • setex
  • get
  • expire

Cache

var redisDriver = require('redis');
var redis = redisDriver.createClient();

redis.setex("expires", 1, "seconds", function (err) {
  redis.get("expires", function (err, val) {
    console.log(val);
  });

  setTimeout(function () {
    redis.get("expires", function (err, val) {
      console.log(val);
      redis.end();
    });
  }, 1500);
});

seconds
					null

Cache

• Scale Up
• Scale Out
• Useful for session data
  • Express-session & connect-redis

Event Emitter

• Design Pattern
  • Messages broadcast to zero or more listeners
  • Possible for message to be consumed more than once
  • Messages do not persist
• Redis Design - Pub/Sub
  • Channels (Event Emitter)
  • Publish (emit)
  • Subscribe (on)
  • More info at redis.io/topics/pubsub

Event Emitter Example

var redis = require("redis");
var subscriber = redis.createClient();
var publisher  = redis.createClient();

subscriber.subscribe("channel 1");
subscriber.subscribe("channel 2");
subscriber.on("message", function (channel, message) {
  console.log("message on %s: %s", channel, message);
});

publisher.publish("channel 1", "Hello Edinburgh.js");
publisher.publish("channel 2", "Hello Edinburgh.js");

message on channel 1: Hello Edinburgh.js
message on channel 2: Hello Edinburgh.js

Event Emitter

• Scale Out
• Does not scale up
  • Each instance will handle message
• Do not dynamically create listeners
  • Memory Leaks
  • once instead of on

Queues

• Design Pattern
  • Jobs are pushed onto a queue
  • Jobs processed once by a worker
  • First in first out (FIFO) order
  • Jobs are persistent
• Redis Design
  • Locks and lists
  • Use a library, bull

Queues Example

var queue = require('bull');
var q = queue('simple queue');

q.process(function (job, done) {
  console.log(job.data);
  done();
});

q.add('a job');

a job

Queues

• Does not scale out
  • Need a queue per application
• Scales Up
• Use case
  • Emails

Advanced Example

Conclusion

• Cache
  • Share volatile data across instances and applications
  • Scales out
  • Scales up
• Event Emitter
  • Share messages across multiple applications.
  • Scales Out
  • Does not scale up
• Queue
  • Offload long running jobs onto scalable workers.
  • Does not scale out
  • Scales Up