Framework-Agnostic Discovery

Tim Gross | Product Manager, Joyent | @0x74696d

Abstract: Moving your application into a container and deploying it to production is a great first step towards taking advantage of containerization. This gets you past "works on my machine", and Docker makes this easy. But the real value of containers -- fast immutable deployments, maximizing resource utilization, and bare-metal performance -- comes from an architecture optimized for containers... a container-native architecture, and requires framework-agnostic discovery

Container-native?

Containers are a first class citizen.

Each container is an equal peer on the network.

Discovery should be framework-agnostic.

Remember: your mission is not "manage VMs."

Your mission is what your application does for your organization.

Infrastructure (undifferentiated heavy lifting) is incidental cost and incidental complexity.

Application containers make the full promise of cloud computing possible...

but require new ways of working.

Triton Elastic Container Service

Run Linux containers securely on bare-metal in public cloud
Or run on-premise (it's open source!)

Director of DevOps

... Docker in production since Oct 2013

What Docker Solved For Us:

Human-and-machine-readable build documentation.

No more "works on my machine."

Fix dependency isolation.

Interface-based approach to application deployment.

Deployments are fast!

DevOps kool-aid for everyone!

Ok, what's wrong?

No obligatory container ship disaster image

NAT

Docker's use of bridging and NAT noticeably increases the transmit path length; vhost-net is fairly efficient at transmitting but has high overhead on the receive side... In real network-intensive workloads, we expect such CPU overhead to reduce overall performance.

IBM Research Report: An Updated Performance Comparison of Virtual Machines and Linux Containers

Can we avoid NAT?

--host networking

port conflicts
port mapping at LB

Can we avoid NAT?

Bridge (not --bridge) networking

Can get IP per container
May need 2nd NIC
Scaling w/ subnet per host

DNS

Simple discovery! But...

Can't address individual hosts behind a record.*

No health checking.*

TTL caching.

Networking still sucks!

Containers don't have their own NIC on the data center network

Pass through proxy for all outbound requests

All packets go through NAT or port forwarding

The Container-Native Alternative?

Cut the cruft!

Push responsibility of the application topology away from the network infrastructure and into the application itself where it belongs.

Responsibilities of a Container

Registration

Self-introspection

Heartbeats

Look for change

Respond to change

No sidecars

Sidecar needs to reach into application container
Unsuited for multi-tenant security
Deployment of sidecar bound to deployment of app

but that means changing app behavior, right? can we avoid changing app if we divest to sidecar?

Application-Aware Health Checks

No packaging tooling into another service

App container lifecycle separate from discovery service

Respond quickly to changes

Legacy Pre-Container Apps

Registration: wrap start of app in a shell script
Self-introspection: self-test?
Heartbeats: um...
Look for change: ???
Respond to change: profit?

http://containerbuddy.io

Containerbuddy:

A shim to help make existing apps container-native

Registration: registers to Consul on startup
Self-introspection: execute user-defined health check
Heartbeats: send health status w/ TTL to Consul
Look for change: poll Consul for changes
Respond to change: execute user-defined response behavior

containerbuddy does this work instead of legacy app

No Supervision

Containerbuddy is PID1

Returns exit code of shimmed process back to Docker Engine (or Triton) and dies

Attaches stdout/stderr from app to stdout/stderr of container

{
  "consul": "consul:8500",
  "services": [
    {
      "name": "nginx",
      "port": 80,
      "health": "/usr/bin/curl --fail -s http://localhost/health",
      "poll": 10,
      "ttl": 25
    }
  ],
  "backends": [
    {
      "name": "app",
      "poll": 7,
      "onChange": "/opt/containerbuddy/reload-nginx.sh"
    }
  ]
}

$ cat ./nginx/opt/containerbuddy/reload-nginx.sh

# fetch latest virtualhost template from Consul k/v
curl -s --fail consul:8500/v1/kv/nginx/template?raw \
    > /tmp/virtualhost.ctmpl

# render virtualhost template using values from Consul and reload Nginx
consul-template \
  -once \
  -consul consul:8500 \
  -template \
  "/tmp/virtualhost.ctmpl:/etc/nginx/conf.d/default.conf:nginx -s reload"

$ less ./nginx/default.ctmpl

# for each service, create a backend
{{range services}}
upstream {{.Name}} {
    # write the health service address:port pairs for this backend
    {{range service .Name}}
    server {{.Address}}:{{.Port}};
    {{end}}
}
{{end}}

server {
    listen 80;
    server_name _;

    # need ngx_http_stub_status_module compiled-in
    location /health {
        stub_status on;
        allow 127.0.0.1;
        deny all;
    }

    {{range services}}
    location /{{.Name}}/ {
        proxy_pass http://{{.Name}}/;
        proxy_redirect off;
    }
    {{end}}
}

nginx:
  image: 0x74696d/containerbuddy-demo-nginx
  mem_limit: 512m
  ports:
  - 80
  links:
  - consul:consul
  restart: always
  environment:
  - CONTAINERBUDDY=file:///opt/containerbuddy/nginx.json
  command: >
    /opt/containerbuddy/containerbuddy
    nginx -g "daemon off;"

echo 'Starting Consul.'
docker-compose -p example up -d consul

# get network info from consul. alternately we can push this into
# a DNS A-record to bootstrap the cluster
CONSUL_IP=$(docker inspect example_consul_1 \
            | json -a NetworkSettings.IPAddress)

echo "Writing template values to Consul at ${CONSUL_IP}"
curl --fail -s -X PUT --data-binary @./nginx/default.ctmpl \
      http://${CONSUL_IP}:8500/v1/kv/nginx/template

echo 'Opening consul console'
open http://${CONSUL_IP}:8500/ui

Starting application servers and Nginx
example_consul_1 is up-to-date
Creating example_nginx_1...
Creating example_app_1...
Waiting for Nginx at 72.2.115.34:80 to pick up initial configuration.
...................
Opening web page... the page will reload every 5 seconds with any updates.
Try scaling up the app!
docker-compose -p example scale app=3

echo 'Starting application servers and Nginx'
docker-compose -p example up -d

# get network info from Nginx and poll it for liveness
NGINX_IP=$(docker inspect example_nginx_1 \
           | json -a NetworkSettings.IPAddress)
echo "Waiting for Nginx at ${NGINX_IP} to pick up initial configuration."
while :
  do
  sleep 1
  curl -s --fail -o /dev/null "http://${NGINX_IP}/app/" && break
  echo -ne .
done
echo
echo 'Opening web page... the page will reload every 5 seconds'
echo 'with any updates.'
open http://${NGINX_IP}/app/

Does it blend scale?

$ docker-compose -p example scale app=3
Creating and starting 2... done
Creating and starting 3... done

The Old Way The Container-Native Way Extra network hop from LB or local proxy Direct container-to-container commmunication NAT Containers have their own IP DNS TTL Topology changes propogate immediately Health checks in the LB Applications report their own health Two build & orchestration pipelines Focus on your app alone VMs Secure multi-tenant bare-metal

http://0x74696d.com/talk-kubecon-2015/

Framework-Agnostic Discovery – NAT

tgross

Framework-Agnostic Discovery – NAT

0 0

talk-kubecon-2015

Framework-Agnostic Discovery

Container-native?

Remember: your mission is not "manage VMs."

What Docker Solved For Us:

Ok, what's wrong?

NAT

NAT

Can we avoid NAT?

Can we avoid NAT?

DNS

Networking still sucks!

The Container-Native Alternative?

Responsibilities of a Container

No sidecars

Application-Aware Health Checks

Legacy Pre-Container Apps

Containerbuddy:

No Supervision

Does it blend scale?

Framework-Agnostic Discovery – NAT

tgross

Framework-Agnostic Discovery – NAT

0 0 (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/platform.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

talk-kubecon-2015

Framework-Agnostic Discovery

Container-native?

Remember: your mission is not "manage VMs."

What Docker Solved For Us:

Ok, what's wrong?

NAT

NAT

Can we avoid NAT?

Can we avoid NAT?

DNS

Networking still sucks!

The Container-Native Alternative?

Responsibilities of a Container

No sidecars

Application-Aware Health Checks

Legacy Pre-Container Apps

Containerbuddy:

No Supervision

Does it blend scale?

0 0