The Twelve-Factor App

Methodology for building saas apps

Presented by @aymericbrisse

Semantic Web & Ruby enthusiast

Objectives

• Raise awareness of some systemic problems of modern application development
• Provide a shared vocabulary for discussing those problems
• Offer a set of broad conceptual solutions to those problems

I. Codebase

One codebase tracked in revision control, many deploys

• Always use a version control system to track the code (Git, Mercurial, Subversion)
• One codebase per application
• No multiple apps sharing the same code
• Multiple deploys per application
• Same codebase accross all deploys, multiple versions

II. Dependencies

Explicitly declare and isolate dependencies

• Each language offers a packaging system for librairies (CPAN, Rubygems, etc.)
• Never rely on implicit existence of system-wide packages
• Use a dependency declaration manifest (bundler, pip)

  bundle install

• Use a dependency isolation tool - (bundler, virtualenv)

  bundle exec

• Simplifies setup for new developers
• Never rely on implicit existence of any system tools (vendoring)

III. Config

Store config in the environment

• Config is what can change between deploys (credentials, services hostname, etc.)
• Strict separation of config from code (config change, code does not): do not store config as constants in code
• Store config in environment variables
• They are language / framework / OS-agnostic (unlike custom files or Java System Properties)
• Avoid config grouping (development/staging/production), it does not scale

IV. Backing Services

Treat backing services as attached resources

• 2 kinds of backing service (any service the app consumes over the network):
  • locally-managed services (e.g. MySQL, RabbitMQ)
  • third-parties services (e.g. New Relic, Amazon S3)
• Make no distinction between them
• Both are attached resources, accessed via a URL or other locator/credentials stored in the config

V. Build, release, run

Strictly separate build and run stages

• 3 stages to transform a codebase into a deploy:
  • The build stage (compile binaries/assets of a given commit, fetch dependencies)
  • The release stage (combine the build with the env config)
  • The run stage (run processes of a given release)
• Use strict separation between these stages
• Deployment tools typically offer release management tools (rollback)
• Unlike the build, the run stage must be kept simple

VI. Processes

Execute the app as one or more stateless processes

• Applications are stateless and independent
• Any data that needs to persist must be stored in a stateful backing service
• The memory space or filesystem of the process can be used as a brief, single-transaction cache.
• Use time-expiration databases for "sticky sessions"

VII. Port binding

Export services via port binding

• Web apps are sometimes executed inside a webserver container (e.g. mod_php, Tomcat)
• Develop completely self-contained apps that exports HTTP as a service by binding to a port
• Use dependency declaration to add a webserver library to the app (e.g. Tornado, Puma, Jetty)

upstream api {
  server 172.17.0.1:5000; # application port binding
  server 172.17.0.2:5000;
}

server {
  listen 80;
  server_name api.clermontech.org;
  root /usr/local/share/api;

  location / {
    proxy_pass http://api; # match the name of upstream directive which is defined above
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

						

VIII. Concurrency

Scale out via the process model

• Use unix process model to run server-side programs (upstart, launchd)
• Architect your app to handle diverse workloads by assigning each type of work to a process type:
  • HTTP requests are handled by a web process
  • Long-running background tasks handled by a worker process
• Adding more concurrency is a simple and reliable operation (share-nothing nature)

IX. Disposability

Maximize robustness with fast startup and graceful shutdown

• Processes are disposable: can be started or stopped anytime
• Minimize startup time
• Graceful shutdown with SIGTERM:
  • Web processes refuse new requests, wait for current requests to finish and exit
  • Worker processes return the current job to the work queue (reentrancy, idempotence)
• Process are robust against sudden death

X. Dev/prod parity

Keep development, staging, and production as similar as possible

• Historically, 3 gaps between development and production:
  • The time gap
  • The personnel gap
  • The tools gap
• Continuous deployment to keep these gaps small Traditional app Twelve-factor app Time between deploys Weeks Hours Code authors vs code deployers Different people Same people Dev vs production environments Divergent As similar as possible
• Resists the urge to use different backing services between development and production
• Chef, Puppet provisioning tools + Virtual env with Vagrant (Docker, Virtualbox)

XI. Logs

Treat logs as event streams

• Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services
• Apps never concern themselves with routing or storage of its output stream
• Each running process writes its event stream, unbuffered, to stdout
• Routing via open-source log routers (Logplex and Fluent)
• Storage via files or log indexing and analysis systems (Splunk, Hadoop/Hive)

XII. Admin processes

Run admin/management tasks as one-off processes

• One-off administrative or maintenance tasks for the app:
  • Running database migrations

    rake db:migrate
    manage.py syncdb
    

  • Running a console (a REPL shell) to run arbitrary code or inspect the app’s models against the live database.

    irb
    rails console
    python

  • Running one-time scripts committed into the app’s repo

    php scripts/fix_bad_records.php

• Thoses processes should be run in an identical environment, i.e. same codebase and config
• With the same dependency isolation techniques

  bundle exec rake db:migrate
  bin/python manage.py syncdb