Architectural style

Microservices

Martin Fowler puts it best:

Brings to the architecture:

Polyglot programming

Polyglot persistence

Best tool for each job

Pressure on the Ops:

Automation

Monitoring

Provisioning

Fast delivery

Archtectural style

REST

REST

• REpresentational
• State
• Transfer

a.k.a: WEB

Resumindo

From Roy Fielding:

• Client-Server
• Cache(able)
• Stateless
• Uniform Interface
• Layered Systems
• Code on-Demand

In practice: HTTP

• Make good use of the headers
• Stateless
• Cacheable (E-Tags)
• URI as identifier
• Verbs as actions (GET, POST, PUT...)
• Formats (content-types):
  • HTML
  • JSON
  • XML
  • CSV

GIT

Solves

• Who has the last version?
• Who changed this line?
• How do I know what changed?
• How to send code?
• I want to try this differently...
• And more.

PHP

The language of the web

Pros

• I can code in PHP
• You can code in PHP
• Lots of resources
• Wordpress, Drupal, Symfony, Facebook...
• Lately has been shaping up good

Cons

• Not all the community is mature
• Easy to make a mess
• Require discipline

Best now a days:

• Namespaces, classes and PDO
• Composer
• PHPUnit or SimpleTest
• Behat (BDD)
• Components from Symfony ou Zend
• Lavarel or Slim
• Templates mustache
• PHPTheRightWay.com

Ruby

The new language of the Web!

ok, nodejs is the new-new, but really...

Pros

• I can code in ruby
• Lot of new programmers can
• Really easy to learn
• Truly Object Oriented
• Great for the web
• Very popular and with modern tools
• And is ok for ops tasks and scripts

Cons

• Sometimes lacks standart on the community
• Easy to make a mess
• Lot's on decision on webserver
• Easy to kill performance(but not that bad really)

How I use it

• 2.1
• Bundler
• RSpec
• Sinatra
• Mustache
• net & json native

Clojure

Functional programming on the JVM

Prós

• JVM and java interop.
• LISP
• Imutability == predictability
• Code is data, data is code
• Parallelism and concurrency
• Java performance, if needed.
• Great standart and great community

Cons

• Learning curve
• Hard to find programmers
• JVM size and startup

How I use it

• lein
• midje
• ring
• compojure
• immutant
• core.async
• prismatic libs
• clj-http + data.json

CouchDB

It is

• REST
• Schemaless
• Documet-based
• Changes Feed
• Incremental
• Efficient HTTP-based replication
• Most trustable DB I have used
• CON: hard/limited to query (map/reduce)

ElasticSearch

You know, for search!

It is

• REST
• Schemaless
• Document-based
• Efficient and flexible searching of anything
• Easy to send data in
• CON: Not your primary database

PostgreSQL

The great SQL DB.

It is

• Relational: everybody can handle it
• Lots of tools
• PostGIS!
• hstore/jsonstore
• Foreign Data Wrappers
• If going relational, go PostgreSQL

SQLite

Default simple SQL DB

It is

• Relational: everybody can handle it
• Faster them it seems
• Lots of read, few parallels writes
• Single file DB, no depency and simple OPS
• WAL, Fulltext-search, SpatialIndex...

TDD

We all knwo this.

BDD

Most of us knows: behavior tests.

Feature: Open and edit profiles

    Scenario: create profile
        Given I am on "/specie/Aphelandra longiflora"
        Then I login as "Diogo", "diogo@cncflora.net", "admin,analyst"
        And I press "create-btn"

    Scenario: Edit a profile, changes apply and metadata
        Given I am on "/specie/Aphelandra longiflora"
        When I login as "Bruno", "bruno@cncflora.net", "analyst", "ACANTHACEAE"
        And I follow "Editar"
        Then I should see "Contribuidor(es): [Bruno] ; Diogo"
        Then I should see "Taxonomic Notes"
        Then I fill field "textarea[id*='-taxonomicNotes-notes']" with "Hello, notes."
        And I press "Salvar"
        Then I should see "Status: open"
        Then I should see "Notas Taxonômicas"
        Then I should see "Hello, notes."

Vagrant!

Development environments made easy.

Standart and automation of dev env.

• Single configuration file (Vagrantfile) for each project
• The dev env is part of the code now
• Source control
• Publish, share and replicate
• Standart

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty32"

  config.vm.network "private_network", ip: "192.168.50.188"

  config.vm.provision :shell, :inline => "apt-get update && apt-get install openjdk-7-jdk curl git tmux vim htop redis-server -y"
  config.vm.provision :shell, :inline => "wget https://raw.github.com/technomancy/leiningen/stable/bin/lein -O /usr/bin/lein"
  config.vm.provision :shell, :inline => "chmod +x /usr/bin/lein"
end

That is just ruby...

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/trusty64"

  config.vm.network "private_network", ip: "192.168.50.10"
  config.vm.network :forwarded_port, host: 8888, guest: 80, auto_correct: true

  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
    v.cpus = 2
  end

  config.vm.provision "docker" do |d|
    d.run "coreos/etcd", name: "etcd", args: "-p 8001:80 -p 4001:4001"
  end

  config.vm.provision "chef_solo" do |chef|
    chef.add_recipe "apache"
    chef.add_recipe "couchdb"
  end
end

But really, docker-compose is better for this.

Docker

Build, ship and run!

It is

• Single Dockerfile by project
• Dev and prod is the same, and part of the code
• Dev and Prod gets version controlled, automated and testable
• Fast

How?

You will automate, but:

# At your machine
docker build -t cncflora/profiles . # Build this project image
docker push cncflora/profiles # Send it to repository
# On the server
docker pull cncflora/profiles # pull lastest image, incrementally
docker run -d -p 8080 -t cncflora/profiles # Runs it
            

Any server, same thing:

• AWS
• Google compute engine
• VPS
• Vmware, xen
• openstack, openshift, cloudfoundry
• Anything Linux

Dockerfile

FROM dockerfile/java

RUN wget http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-runner/9.2.0.M0/jetty-runner-9.2.0.M0.jar -O /root/jetty.jar

ADD target/dwc-services-0.0.4-standalone.war /root/dwc-services.war

EXPOSE 8080

CMD ["java -jar","/root/jetty.jar","/root/dwc-services.war"]

Dockerfile

FROM ubuntu:14.04

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install apache2 libapache2-mod-php5 php5 php5-cli php5-curl php5-common php5-sqlite php5-mysql php5-pgsql php5-gd -y 

RUN apt-get install supervisor -y
RUN mkdir /var/log/supervisord 

RUN a2enmod rewrite 

RUN sed -i -e 's/memory_limit.*/memory_limit=512M/g' /etc/php5/apache2/php.ini && \
    sed -i -e 's/upload_max_filesize.*/upload_max_filesize=128M/g' /etc/php5/apache2/php.ini && \
    sed -i -e 's/post_max_size.*/post_max_size=128M/g' /etc/php5/apache2/php.ini && \
    sed -i -e 's/display_errors.*/display_erros=On/g' /etc/php5/apache2/php.ini

ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ENV APACHE_PID_FILE /var/apache2.pid
ENV PHP_ENV production

ADD default.conf /etc/apache2/sites-available/000-default.conf

ADD supervisord.conf /etc/supervisor/conf.d/apache.conf

ADD vendor /var/www/vendor
RUN chown www-data.www-data /var/www/vendor -Rf

ADD . /var/www
RUN chown www-data.www-data /var/www -Rf

EXPOSE 80
EXPOSE 9001

CMD ["supervisord"]

Dev Ops

Beyond docker.

docker-compose

etcd:
    image: coreos/etcd
    name: etcd
    ports:
       - "4001:4001"
logstash:
    image: diogok/logstash
    name: logstash
    ports:
        - 80
        - 9514
proxy:
    image: diogok/proxy-docker
    name: proxy
    environment:
        ROOT_APP: "connect"
    ports:
        - "80:80"
        - "9001"
    links:
        - "etcd:etcd"
connect:
    image: cncflora/connect
    name: connect
    ports:
        - "8080"
        - "9001"
    volumes:
        - "/var/floraconnect:/var/floraconnect:rw"
    environment:
        PROXY: "/connect"
dwc_services:
    image: cncflora/dwc-services
    name: dwc_services
    ports:
        - "8080"
        - "9001"
    environment:
        PROXY: "/dwc_services"
floradata:
    image: cncflora/floradata
    name: floradata
    ports:
        - "80"

Orchestrate your container.

ETCD

Share configuration REST server.

Nginx

Proxy to the container network

http {
    server {
        listen 80;

        # Proxying the connections connections, data come from ETCD
        <% for @app in @data.values %>
        location /<%= @app['name'] %>/ {
            rewrite /<%= @app['name'] %>/(.*) /$1 break;
            proxy_pass         http://<%= @app['networksettings']['ipaddress'] %>:<%= @app['networksettings']['ports'].keys.first.gsub("/tcp","") %>;
            proxy_redirect     default;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Host $server_name;
        }
        <% end %>
    }
}

Supervisord

SUpervvisor the processes

Logstash

Get those logs in a nice view

Next step:

Continuous integration

Continuous delivery

TODO