intro-restful-api-slides

"REpresentational State Transfer" (REST)

First described by Roy Fielding (in his dissertation)

Essentially describes the way the web works

REST is a set of principles that define how Web standards, such as HTTP and URIs, are supposed to be used (which often differs quite a bit from what many people actually do).Stefan-Tilkov (post)

Navigate 'down' for some background on REST, or 'right' to continue

REST is "Simple"

REST does NOT propose an alternative to the web(like SOAP/WS-*, CORBA, RMI)

REST provides a common and consistent interface based on proper use of HTTP

Navigate 'down' for more info on REST, or 'right' to continue

Uses Standard HTTP Methods

*Notes:

• PUT is often used with PATCH semantics.
• PUT can create a thing if it has a pre-assigned id.

Uses Standard HTTP Status Codes

Status Codes continued...

It's all about 'resources'

"A resource is anything that's important enough to be referenced as a thing in itself." Richardson and Ruby

• 'resource orientation'
  • thinking about solutions in terms of resources
  • may significantly influence internal design & architecture
  • REST principles have been shown to reduce complexity
• 'resources' often correspond to persistent domain objects
  • but they don't have to...

And 'representations'

• 'resources' are the concepts, or entities
• 'representations' are how they are presented
  • JSON and XML are most common
  • other representations include iCal, PDF, etc.
• a resource may have many representations

When manipulating a 'resource' through the standard REST actions, it is done by using a representation of the resource

Learning about REST

Blogs and tutorials are everywhere... such as the Rest API Tutorial So... how come so many APIs aren't right?

REST fundamentals are not enough

While REST is "easy", it doesn't answer everything...

• How do I secure my API?
• Should I use JSON? Do I need an envelope?
• What about versioning? How?
• Conditional requests? CORS?

Many frameworks provide support that is naive

• promote coupling to domain objects
• don't support versioning

Exposing a Resource

Routing Requests

A single controller may be used to handle all API requests

class UrlMappings {

  static mappings = {

    // Normal REST mappings

    "/api/$pluralizedResourceName/$id"(controller:'restfulApi') {
        action = [GET: "show", PUT: "update", DELETE: "delete"]
        parseRequest = false
    }
    "/api/$pluralizedResourceName"(controller:'restfulApi') {
        action = [GET: "list", POST: "create"]
        parseRequest = false
    }

Routing Requests

We recommend explicit support for one level of nesting

class UrlMappings {

  static mappings = {

    // Also add mappings to support nesting resources one-level

    "/api/$parentPluralizedResourceName/$parentId/$pluralizedResourceName/$id"(controller:'restfulApi') {
      action = [GET: "show", PUT: "update", DELETE: "delete"]
      parseRequest = false
    }

    "/api/$parentPluralizedResourceName/$parentId/$pluralizedResourceName"(controller:'restfulApi') {
      action = [GET: "list", POST: "create"]
      parseRequest = false
    }

Routing Requests

And if you need to query you can also use POST(useful for complex queries or when criteria is private data)

class UrlMappings {

  static mappings = {

    // Using a different URL prefix, we'll add a mapping to
    // support querying using POST

    "/qapi/$pluralizedResourceName"(controller:'restfulApi') {
      action = [GET: "list", POST: "list"]
      parseRequest = false
    }

Whitelisting Resources

Configuration may be used to 'whitelist' exposed resources

resource 'things' config {
    representation {
        mediaTypes = ['application/vnd.hedtech.v1+json',
                      'application/json']
        marshallerFramework = 'json'
        marshallers {
            jsonDomainMarshaller {
                priority = 100
            }
        }
        jsonExtractor {}
    }
}

Dynamically Exposing Resources

...or you can dynamically expose all services

anyResource {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                priority = 101
            }
            jsonGroovyBeanMarshaller {
                priority = 100
            }
        }
        jsonExtractor {}
    }
}

Processing a Request

Leveraging HTTP

The plugin ensures proper, consistent HTTP support in accordance with the strategy

• Support 'GET', 'POST', 'PUT', 'DELETE', 'OPTIONS' HTTP Methods
  • Note: Currently, 'PUT' support employs 'PATCH' semantics
• Use standard HTTP Headers
• Support conditional requests
• Support CORS (Cross-Origin Resource Sharing)

Content Negotiation

Parse request body based on 'Content-Type' header

Respond with format identified in 'Accept' header

Given:

        application/xml;q=0.9,application/vnd.hedtech.v0+xml;q=1.0

Will use:

        application/vnd.hedtech.v0+xml

• Media Type Support
  • Built-in support for JSON and XML representations
  • Can add support for others (iCal, PDF)
  • Versioning is supported through custom media types

Content Extraction

Controller will delegate to a configured extractor to process a request body into a map before passing it to a service.

 resource 'customers' config {
     representation {
         mediaTypes = ["application/json"]
         extractor = new net.hedtech.restfulapi.CustomerExtractor()
     }
 }

Content Extraction (continued...)

Three types of extractor interfaces available. Each is responsible for returning a map of properties that the service will use to fulfill the request

• JSONExtractor - passed a JSONObject parsed by grails JSON converter
• XMLExtractor - passed a GPATHResult object parsed by grails XML converter
• RequestExtractor - passed the http request directly

Use the RequestExtractor to get access to the request body directly. Used to support alternate data-binding frameworks (google-gson, JAXB, etc).

Declarative Extraction

Can define rules in configuration to extract content from JSON and xml.

• Rename properties
• Provide default values
• Convert 'short-object' reference to id
• 'Flatten' maps to be compatible with grails data-binding

Declarative Extraction (continued...)

    resource 'purchase-orders' config {
        representation {
            mediaTypes = ["application/json"]
            jsonExtractor {
                property 'productId'      name 'productNumber'
                property 'customers.name' name 'lastName'
                property 'orderType'      defaultValue 'standard'
            }
        }
    }
`

Applied to input

    {
        "productId":"123",
        "quantity":50,
        "customers":[
            {"name":"Smith"},
            {"name":"Jones"}
        ]
    }

Results in the map

['productNumber':'123', 'quantity':50, 'orderType':'standard',
 customers':[['lastName':'Smith'], ['lastName':'Jones'] ]

Declarative Extraction - flattening maps

Can flatten maps for compatibility with grails data binding

    resource 'purchase-orders' config {
        representation {
            mediaTypes = ["application/json"]
            jsonExtractor {
                property 'customer' flatObject true
            }
        }
    }

Applied to

    {"orderId":123,
     "customer": {
        "name":"Smith"
        "id":456,
        "phone-number":"555-555-5555"
     }
    }

Will result in the map

['orderId':123, 'customer.name':'Smith', 'customer.id':456,
 'customer.phone-number':'555-555-5555']

Delegating to a Service

The controller delegates to a service based on naming conventions (or configuration)

    /course-sections/2351  -->  CourseSectionService

Establishes a contract for services

def list(Map params)
def count(Map params)
def show(Map params)
def create(Map content, Map params)
def update(def id, Map content, Map params)
void delete(def id, Map content, Map params)

(but you may configure a 'ServiceAdapter' Spring bean to adapt the contract)

Filtering Lists

Filter lists using query parameters or within a POST body

?filter[0][field]=description&filter[1][value]=6322&
filter[0][operator]=contains&filter[1][field]=thing&
filter[1][operator]=eq&filter[0][value]=science&max=50

Helper class may be used within your service to generate HQL (but it is not a sophisticated query engine)

def query = HQLBuilder.createHQL( application, params )
def result = Thing.executeQuery( query.statement, query.parameters,
                                 params )

'POST' queries should use a separate URI (e.g., '/qapi' vs. '/api')

Handling Exceptions

Built-in exception handling for:

• OptimisticLockException
• Validation Exception
• UnsupportedResourceException
• UnsupportedRequestRepresentationException
• UnsupportedResponseRepresentationException
• ApplicationException*
• AnyOtherException

*'ApplicationException' allows your services to specify how an exception should be handled.

Responding

Leveraging HTTP

The plugin ensures proper, consistent HTTP support in accordance with the strategy

• Return proper HTTP Status Codes
• Use standard and custom HTTP Headers
  • 'Content-Type' specifies standard media type (to allow tools, browsers to render result)
  • A custom header (e.g., 'X-hedtech-Media-Type') identifies actual media type of response
• Include Cache Headers (ETag, Last-Modified)

The Response Body

Response bodies contain either:

• a single representation
• an array of representations
• an empty body

'Envelope' information is contained in headers

Content-Type
X-hedtech-Media-Type   <-- the 'actual' content type
X-hedtech-message      <-- localized message
X-Status-Reason        <-- Optionally returned w/ 400 response
ETag
Last-Modified
X-hedtech-totalCount   <-- paging
X-hedtech-pageOffset
X-hedtech-pageMaxSize

Marshalling

• Leverages the grails converters and priority mechanism
• Ultimate flexibility
  • can write custom marshallers
  • can delegate to a custom marshalling service and use another framework (google-gson, JAXB, XMLBeans, etc)

Declarative marshalling

• For most cases, writing a custom marshaller shouldn't be needed
• Can declare how to marshall a class in configuration
• Supports
  • including all fields except those excluded
  • including only explicity listed fields
  • renaming fields in the representation
  • declaring additional fields to include in the representation, such as affordances
  • declaring how to represent persistent associations

Example of declaring what fields to include

resource 'customers' config {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                includesFields {
                    field 'firstName'
                    field 'lastName'
                    field 'customerNo' name 'customerID'
                }
            }
        }
    }
}

{"id":74,"version":0,"firstName":"John",
 "lastName":"Smith","customerID":"12345"}

Declarative affordances

• Can inject affordances or other computed fields into representations declaratively, in a re-usable fashion (using template)

        jsonDomainMarshallerTemplates {
            template 'domainAffordance' config {
                additionalFields { map ->
                    map['json'].property("_href",
                        "/${map['resourceName']}/${map['resourceId']}" )
                }
            }
        }

Declarative affordances (continued...)

resource 'customers' config {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                inherits = ['domainAffordance']
                includesFields {
                    field 'firstName'
                    field 'lastName'
                    field 'customerNo' name 'customerID'
                }
            }
        }
    }
}

{"id":74,"version":0,"firstName":"John","lastName":"Smith",
 "customerID":"12345","_href":"/customers/74"}

Deep vs Shallow Marshalling of associations

• declarative domain marshallers support both deep or 'short-object' marshalling

  resource 'purchase-orders' config {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                includesFields {
                    field 'poNumber'
                    field 'customer' deep false
                }
            }
        }
    }
  }

  {"id":74,
  "version":0,
  "poNumber":12345,
  "customer":"/customers/123"
  }

Deep vs Shallow Marshalling (continued...)

resource 'purchase-orders' config {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                supports PurchaseOrder
                includesFields {
                    field 'poNumber'
                    field 'customer' deep true
                }
            }
            jsonDomainMarshaller {
                supports Customer
                includesVersion false
                includesFields {
                    field 'lastName'
                    field 'firstName'
                    field 'phone'
                }
            }
        }
    }
}

{"id":74,
 "version":0,
 "poNumber":12345,
 "customer": {
        "id":123
        "lastName":"Smith"
        "firstName":"John"
        "phone":"555-555-5555"
    }
}

Deep vs Shallow Marshalling (continued...)

• default format for a 'short-object' can be changed in configuration
• can apply to all representations via template inheritance

Reusing marshallers

• Marshallers that need to be reused (such as a deep-marshalled sub-object) can be declarared in re-usable groups

  marshallerGroups {
    group 'customer' marshallers {
        jsonDomainMarshaller {
            supports Customer
            includesFields {
                field 'lastName'
                field 'firstName'
                field 'phone'
            }
        }
    }
  }

Reusing marshallers (continued...)

resource 'purchase-orders' config {
    representation {
        mediaTypes = ["application/json"]
        marshallers {
            jsonDomainMarshaller {
                supports PurchaseOrder
                includesFields {
                    field 'poNumber'
                    field 'customer' deep true
                }
            }
            marshallerGroup 'customer'
        }
    }
}

Declarative support for both domain and POGOs

• jsonDomainMarshaller
• jsonGroovyBeanMarshaller
• xmlDomainMarshaller
• xmlGroovyBeanMarshaller

(Support for POJOs coming in the future, if needed.)

Custom marshalling service

• Can mix marshalling frameworks in the same application - use grails converters for some objects, custom service(s) for others
• A given representation must marshall all objects (and subobjects) using a single framework
• Any bean implementing a marshalObject method can be used

Object marshalObject(Object o, RepresentationConfig config)

Custom marshalling service (continued...)

• marshalObject can return any of the following:
  • String
  • byte[]
  • InputStream
  • StreamWrapper (used to convey both an InputStream and the value of the Content-Length header)
• Allows for marshalling to binary representations (pdf, compression of large results downloaded as a file)

Custom marshalling service - iCalendar example

Use ical4j library to create and marshall calendars

import net.fortuna.ical4j.model.*
import net.fortuna.ical4j.model.property.*
class CalendarService{
    def show( Map params ) {
        def builder = new ContentBuilder()
        def calendar = builder.calendar() {
            prodid('-//John Smith//iCal4j 1.0//EN')
            version('2.0')
            vevent() {
                uid('1')
                dtstamp(new DtStamp())
                dtstart('20090810', parameters: parameters() {
                    value('DATE')})
                action('DISPLAY')
                attach('http://example.com/attachment', parameters: parameters() {
                    value('URI')})
            }
        }
        calendar.validate()

        calendar
    }
}

Custom marshalling service - iCalendar example (continued...)

import groovy.xml.MarkupBuilder
import net.hedtech.restfulapi.config.RepresentationConfig

/**
 * A demonstration class for custom marshalling of iCalendar objects.
 * In this case, we are using ical4j, so we only need to invoke
 * toString on the passed objects.
 */
class ICalendarMarshallingService {

    @Override
    String marshalObject(Object o, RepresentationConfig config) {
        if (!(o instanceof net.fortuna.ical4j.model.Calendar)) {
            throw new Exception("Cannot marshal instances of" + o.getClass().getName())
        }
        return o.toString()
    }
}

Custom marshalling service - iCalendar example (continued...)

Define the calendar resource and representation using the custom marshalling service

resource 'calendars' config {
    methods = ['show']
    representation {
        mediaTypes = ['text/calendar']
        contentType = 'text/calendar'
        marshallerFramework = 'ICalendarMarshallingService'
    }
}

Custom marshalling service - iCalendar example (continued...)

curl -i --noproxy localhost -H "Accept: text/calendar" http://localhost:8080/test-restful-api/api/calendars/1

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
ETag: c44e6bb0-703f-4b39-9f5c-627aedbebc71
Last-Modified: Wed, 24 Jul 2013 15:04:32 GMT
X-hedtech-Media-Type: text/calendar
X-hedtech-message: Details for the calendar resource
Content-Type: text/calendar;charset=utf-8
Transfer-Encoding: chunked
Date: Wed, 24 Jul 2013 15:04:32 GMT

BEGIN:VCALENDAR
PRODID:-//John Smith//iCal4j 1.0//EN
VERSION:2.0
BEGIN:VEVENT
UID:1
DTSTAMP:20130724T150432Z
DTSTART;VALUE=DATE:20090810
ACTION:DISPLAY
ATTACH;VALUE=URI:http://example.com/attachment
END:VEVENT
END:VCALENDAR

Current Status

• Production quality, 0.8.0 release (4 Nov 2013)

Grab it here

Git Repo: https://github.com/restfulapi/restful-api (git clone https://github.com/restfulapi/restful-api.git)

CI Server: (Currently using Jenkins within Ellucian; we'll try to move this soon)

README: https://github.com/restfulapi/restful-api/blob/master/README.md

Strategy: https://github.com/restfulapi/api-strategy/blob/master/README.md

JIRA: (Currently using JIRA within Ellucian)