The Little Schema – Stop reinventing wheels – Enter JSON Schema!



The Little Schema – Stop reinventing wheels – Enter JSON Schema!

0 0


the-little-schema

A presentation about JSON schemas

On Github davidbanham / the-little-schema

The Little Schema

With David Banham

Schemaless data isn't

NoSQL databases are usually schemaless.

That doesn't mean your data is.

It just means the database won't help you with it.

Data always has a schema. Even if it only exists informally in a programmer's head, it's still a schema. It's just a terrible schema that nobody else can use. This doesn't mean NoSQL databases are bad. It can be really useful to have the schema validation stuff separate from your data storing stuff. You just have to make sure you're actually doing both things.

Stop reinventing wheels

It's 2015. We validate data on the client side in our rich web applications. We pass that data back to one (or more) backend services. We then store that data in a database. We're probably validating that data in three different ways in each of these places. That's properly nuts.

Enter JSON Schema!

Just the best marketing website ever. The first draft was published by the IETF in 2009. We're currently on draft v4 which was published in Jan 2013.
"persisted_at": {
  "description": "When the document was persisted to the database.",
  "example": "2012-01-01T12:00:00Z",
  "format": "date-time",
  "type": "string"
}
            
Simple definition of an ISOString() timestamp. There are a few built-in format verifiers that come with regexes.
"email": {
  "description": "Email address of the member",
  "example": "nickjones@example.com",
  "format": "email",
  "type": "string",
  "maxLength": 100
}
            
Email is another inbuilt type, which is handy. This example also shows a maxLength property. It's trivial to mix and match validations so suit your needs.
"card_type": {
  "type": "string",
  "example": "Mastercard",
  "options": [
    "VISA",
    "Mastercard",
    "AMEX",
    "Diners"
  ]
}
            
Arrays of options are very handy. There are lots of other validation types. Using multipleOf to catch floating point errored numbers is a favourite of mine. JSON Schema is a really rich, well thought-out validation system. It's way better than the random collection of validation snippets you've probably hacked together over time.

Now for Hyper Schemas!

"description": "A Transaction occurs when an order is placed or a refund is processed.",
"links": [
  {
    "description": "Create a new transaction.",
    "href": "/transaction",
    "method": "POST",
    "rel": "create",
    "schema": {
      "required": [
        "order_id",
        "category",
        "member",
        "transaction_date",
        "product_description",
        "payment",
        "points",
        "cinema",
        "vouchers",
        "commission",
        "delivery"
      ],
            
Hyper schemas are a superset of JSON schemas. They take many JSON schemas and describe how they should be composed into HTTP APIs. You get methods, required and optional variables, examples, expected responses, etc etc.

Now you have super powers.

* Automatic generation of documentation

* Automatic server-side validations of input from clients

* Automatic generation of API client libraries in multiple languages

* Simpler design of the API by making inconsistencies easier to spot

Source: Auto Generating a Go API client for Heroku

This is just the list of benefits casually tossed off by Heroku when they documented their API with a hyper schema. Automatic generation of API client libraries in multiple languages. That is _impressive_. Not even mentioned is the awesome validation toolchain you get for free.

Docson

http://lbovet.github.io/docson/index.html#/docson/examples/example.json Docson is a documentation tool for JSON schemas. It's a pretty thing that you get totally for free. It just generates this.

prmd

prmd is a documentation tool for JSON hyper schemas. It also has some handy tooling to generate boilerplate for you. This is way better documentation than I would ever write by hand. All this stuff would be a nightmare to keep manually updated. It's just done for you.

More goodies!

This is the benefit that standards bring us. We all work on the web. We all know that life is better when things like browsers are standards-compliant. This is the same principle.

Get involved

https://github.com/Prismatik/document_migrator https://github.com/Prismatik/document_validator We're taking this stuff to it's logical conclusion. Tools like lazily evaluated database migrations. You should be able to modify your database schema without having to take your app down while you modify every single record in your database. Especially if you actually have a lot of data. You also shouldn't have to be handcuffed by how your data looked years ago when developing a feature today. We're building tooling to help with that. If that sounds fun, we'd love to have you on board.

More information

Understanding JSON schema