I will talk tonight on 'JS on the LEVEL'. A high level preso on streams, why they are good, and some helpful tools

What is LevelDB?

• Embedded key/value data store.
• Based on ideas in Google’s BigTable
• Used as the backing-store for Chrome’s IndexedDB

Features

• Arbitrary byte arrays Can anything from ASCII strings to binary blobs.
• Compressed storage Google’s Snappy compression
• Sorted by keys One of the main distinguishing features, very useful for querying as we’ll see later.
• Bi-directional iterators Iteration can move forward and backwards through the store.

Why is it called leveldb?

Writes are written to a level, when its full, a new one is created above it, pushing down older data.

0 ↠ 4 SST, 1 ↠ 10M, 2 ↠ 100M, 3 ↠ 1G, 4 ↠ 10G, 5 ↠ 100G, 6 ↠ 1T, 7 ↠ 10T

NODE BASEA New Database Paradigm

npm install level --save

Get and Put

var level = require('level')

// 1) Create our database
var db = level('./mydb')

// 2) put a key & value
db.put('ryan', 'happy', function (err) {
  if (err) return console.log('Ooops!', err) // some kind of I/O error

  // 3) get by key
  db.get('name', function (err, value) {
    if (err) return console.log('Ooops!', err) // likely the key was not found

  })
})

All operations are asynchronous although they don't necessarily require a callback

Delete

db.del(key[, callback])

Batch

var ops = [
    { type: 'del', key: 'father' }
  , { type: 'put', key: 'name', value: 'Yuri Irsenovich Kim' }
  , { type: 'put', key: 'dob', value: '16 February 1941' }
  , { type: 'put', key: 'spouse', value: 'Kim Young-sook' }
  , { type: 'put', key: 'occupation', value: 'Clown' }
]

db.batch(ops, function (err) {
  if (err) return console.log('Ooops!', err)
  console.log('Great success dear leader!')
})

Provide a list of operations to be executed sequentially, although as a whole they are performed as an atomic operation inside LevelDB.

Batch (fluent form)

db.batch()
  .del('father')
  .put('name', 'Yuri Irsenovich Kim')
  .put('dob', '16 February 1941')
  .put('spouse', 'Kim Young-sook')
  .put('occupation', 'Clown')
  .write(function () { console.log('Done!') })

Get a lightsabre

Streams!

function copy (srcdb, dstdb, callback) {
  srcdb.createReadStream()
    .pipe(dstdb.createWriteStream())
    .on('close', callback)
}

db.createReadStream()
  .on('data', function (data) {
    console.log(data.key, '=', data.value)
  })
  .on('error', function (err) {})
  .on('close', function () {})
  .on('end', function () {})

Lightsaber Slicing

db.createReadStream({start: 'r', end: 'r' + '\xff'})
  .on('data', function (data) {
    only keys that start with r
  })

• start default it will start at the beginning of store. *start* doesn't have to be an actual key that exists, LevelDB will simply find the *next* key, greater than the key you provide.
• end default it will continue until the end of store. *end* doesn't have to be an actual key as an (inclusive) `<=`-type operation is performed to detect the end.
• reverse a boolean, set to true if you want the stream to go in reverse order. A reverse seek will be slower than a forward seek.
• limit The *maximum* number of results, and may not be reached if you get to the end of the store or your `'end'` value first.

node-levelup

LevelUP becomes the API. No backing store

node-leveldown

Pure C++ Node.js LevelDB binding serving as the back-end to LevelUP

Alternate Backends

• MemDown
• MysqlDown
• RiakDown
• RedisDown
• AzureDown

To The Browser!

level.js an implementation of the leveldown API on top of IndexedDB (which is in turn implemented on top of LevelDB, which brings this whole shebang full circle)

This library is best used with browserify

This week the last browser holdout of IndexedDB was announced!

Use different backend

var localstorage = require('localstorage-down');
var levelup = require('levelup');
var db = levelup('/does/not/matter', { db: localstorage });

Plugins!

Things that you can use with any LevelDown compatable storage.

level-sublevel

Adds the ability to create subsections that present the same API as LevelUP, but only write/read to a prefixed section, or bucket, of the key-space.

var SubLevel = require('level-sublevel')
    , db = level('/somewhere')
    , sub = SubLevel(db)
    , people = sub.sublevel('people')
    , places = sub.sublevel('places')
    , things = sub.sublevel('things')

jsonquery-engine

A full MongoDB query language implementation with INDEXES for querying your levelup/leveldb database.

db.query({ $and: [ { tags: 'tag1' }, { num: { $lt: 100 } } ] })
    .on('data', console.log)
    .on('stats', function (stats) {
      // stats contains the query statistics in the format
      //  { indexHits: 1, dataHits: 1, matchHits: 1 });
    });

multilevel-http

A limitation of LevelDB is that only one process is allowed access to the underlying data. multilevel-http exports a LevelDB instance over http.

Server

var multilevel = require('multilevel-http')
// db = levelup instance or path to db
var server = multilevel.server(db, options)
server.listen(5000)

Client

var multilevel = require('multilevel-http')
var db = multilevel.client('http://localhost:5000/')
// now you have the complete levelUP api!

level-replicator

A simple eventually consistent master-master replication module for leveldb.

How 'bout a graph database?

levelgraph

BYODB

The simple api surface area, combined with a powerful module system lets you build a DB as you want it.

Recap

You can use levelup everywhere. Client, server, etc...

Tight API design and the small module philosophy that has been a success for LevelDB

LevelDB will be EVERYWHERE if you do JS

THANKS!

By Ryan Ramage / @ryan_ramage_