Pragmatic RESTful API Design

Chris Dail - @chrisdail

Director, Software Engineering at EMC

http://chrisdail.github.io/talk-pragmatic-rest-api-design/

Disclaimer

This is a pragmatic approach to REST (not a dogmatic one).
Ideas here are simply my opinion. I favour user experience and developer sanity over idealogical 'correctness'.
There is no REST spec or standard (but there is for HTTP)
Enterprise Software and Products vs SaaS

We love to think our products will solve all customer problems but this is not realistic

Why APIs?

Customer's don't want a product; they want a solution to their problem.
APIs allow integrating your product into the customer's solution.
Prevents the feeling of 'lock in'

v1.0 Feature

Most companies discover they need an API long after v1.0
Often an afterthough and maybe not given the attention it needs

Microservices

Distributed applications, Netflix
Applications build with small decoupled, self-contained components
Each component handles a single job
Components communicate with each other via well established APIs

What Kind of API do I need?

What is REST?

Representational State Transfer

“HTTP based RESTful APIs are defined with these aspects: base URI, such as http://example.com/resources/ an Internet media type for the data. This is often JSON but can be any other valid Internet media type (e.g. XML, Atom, microformats, images, etc.) standard HTTP methods (e.g., GET, PUT, POST, or DELETE)” - Wikipedia

Or simply...

Web APIs
Web Services

Why REST?

Universal language, the Web
Universally available protocol, HTTP
Name one programming language that cannot talk to an HTTP server

Elements to a Good API

Features (I'll assume you have this covered)
User Experience - Know your user
Great Documentation

User Experience

External Consistency - Does it work like other APIs
Internal Consistency - Is the API consistent within itself

URLs

Resource Oriented
Nouns not Verbs
Pluralize Resources
Lower case, hypen separated
Short segments (1-2 words)
Top level component (when API gets big)

            /widgets
            /storage-systems     # Storage or Systems is not specific enough
            /foo/things          # 'foo' component allows multiple /things

Query Parameters

Used for optional parameters, options, search, filter, paging
Should never be required - Sensible Defaults
Data format case
Short (1-2 words)

            /resources?type=volume
            /widgets?limit=100
            /foo/things?tenant=coke
            /widgets?q=frank

CamelCase vs snake_case

vi vs emacs
Consistency!

Data Formats

JSON First, XML if required
Case Consistency with all formats
Keep fields short (1-3 words)
Handle Collections Differently

              "things: [
                { },
                { }
              ]
              
              <things>
                <thing></thing>
                <thing></thing>
              </things>

Field Formatting

Numbers as numbers

  // This
  "size_gb": 10

  // Not this
  "size": "10gb"

Dates ISO 8601 (xsd:dateTime, JavaScript standard) yyyy-mm-dd'T'hh[:mm]

Read vs Write data models

Command Query Responsibility Segregation (CQRS)
Separate models for read/write
Example: ID is required for update, but not create
Example: Password required on create, not part of read

HTTP Methods

Create, Read, Update, Delete -> POST, GET, PUT and DELETE

Operation Summary POST /widgets Creates a new Widget GET /widgets Lists widgets GET /widgets/12 Gets a Widget PUT /widgets/12 Updates a widget (whole resource) DELETE /widgets/12 Deletes a widget

HTTP Methods <2>

Operation Summary PATCH /widgets/12 Partial Update of resource GET /widgets/12/knobs Retrieves sub-collection knobs POST /widgets/12/knobs Updates sub-collection knobs POST /widgets/12/action Performs an action on a resource (Verbs here)

Linking

Self-describing relationships and capabilities
HATEOAS - Links as part of data
Link Headers

            GET /widgets/12
            Link: <https://host/v1/widgets/12/knobs>; rel="knobs"

Paging

You can't always return everything (Scalability)
Offset or Marker based
Include links for next/previous/last/first

Query Parameter Example (GET) limit, count, max /widgets?limit=100 offset, index, page /widgets?limit=100&offset=100 marker, since_id, max_id /widgets?marker=id123

Paging - Example

              GET /widgets?limit=100
              Link: </widgets?limit=100&offset=100>; rel="next", 
                    </widgets?limit=100&offset=22300>; rel="last"

              GET /widgets?limit=100
              Link: </widgets?limit=100&marker=id123>; rel="next"

Sorting, Filtering and Searching

Rarely do users want everything

Query Example (GET) sort /widgets?sort=name /widgets?sort=owner,-last_modified q /widgets?q=frank&sort=owner - /widgets?type=floodle /widgets?state=pending&tenant=coke fields /widgets?fields=name,state,type,id

Error Codes

Code Description 200 OK Request processed as expected 201 Created Request created a resource, Set Location Header 202 Accepted Request accepted and pending (Asynchronous), Set Location Header 204 No Content Same as 200 except no content returned 302 Moved Temporarily Standard redirect, Set Location Header 304 Not Modified Cache still good. User requests If-None-Match or If-Modified-Since

Error Codes <2>

Code Description 400 Bad Request Client submitted bad request (Validation) 401 Unauthorized Not authenticated or token expired 403 Forbidden User does not have permission 404 Not Found Resource not found for ID 500 Internal Server Error Server error, should not be retried 503 Service Unavailable Retriable error, typically for connection issues

Versioning

Content NegotiationAccepts: application/vnd.widgets+json; version="1"
URL Basedhttps://hostname/v1/widgets/12

URL Based

Easiest to implement for you
Easiest to implement for users (no headers)
Most widely used in public APIs

Backwards Compatibility

API Change Types

Backwards Compatible
- Additions only. Never remove or rename.
- New fields added to model must be optional
Backwards Incompatible
- Renaming or moving API roots or fields
- Deleting APIs
- Adding Required Fields
- Changes to authentication or headers/cookies
- Changes to behaviour

Versioning Rules

API version should be a positive integer. Ex: v1, v2, v3
Not every product release needs an API version
New version only when backwards incompatible change required
It is easiest to add versioning concepts in v1!

Common Data Formats

Errors
Async Tasks
Links

Error Format

            POST /widgets/12
            {
              "code": 9001,
              "message": "Error updating resource",
              "details": "Error updating resource with ID 12 because of bla"
            }

Error Format (Validation)

            POST /widgets/12
            {
              "code": "9001",
              "message": "Validation Error",
              "details": "Validation error with the hostname field"
              "fields": [
                {
                  "name": "hostname",
                  "code": "2001",
                  "message": "Hostname field not specified but is required"
                }
              ]
            }

Authentication

Always use SSL
Public APIs, use standard SSO like OAuth
HTTP Basic Auth, Auth Token in cookies/headers
Browser Explorable

Caching

ETag - Hash/sum in ETag header. If-None-Match
Last-Modified - Last-Modified header. If-Modified-Since

Random Thoughts

Consolidate APIs under single domain
Support GZip encoding
Consider an SDK for target developers

Documentation

Github and Stackoverflow good examples
Consider generating documentation
Inline testing

Swagger

API descriptor file
Online Editor http://editor.swagger.io
Interactive Documentation http://petstore.swagger.wordnik.com/
Code Generation (Client and Server)
Alternatives: I/O Docs, Carte

Game

What's Wrong With This API
Name some other poor APIs

Pragmatic RESTful API Design – Data Formats – Versioning

chrisdail