docker image much faster than building an AMI
Microservices & The 12 Factor App
Production essentials
- Configuration management
- Deployment
- Logging
- Monitoring
- Debugging
- Security
The purpose of Configuration Management is to establish and maintain
the integrity of work products using configuration identification,
configuration control, configuration status accounting, and configuration
audits. — CMMI-SE/SW, V1.02
bare minimum stuff for capability maturity model integration level 2
puppet + git
processes too, not just tools
...applied to Docker
-
Host level
-
Container level
-
Organisation level
Host level
Bread and butter stuff
- Configure host OS
- Install docker engine
- Start containers
Supported by Puppet,
Chef,
Ansible,
Salt, …
Container level
- Configuration applied during new image builds
- Replace containers, don't upgrade them
- Treat running containers as read-only
Probably masterless via puppet apply.
Organisation level
- Share playbooks between containers and hosts, and between teams and projects
- Slot into existing management and audit processes
Starting simply…
Let's start from first principles.
Dev
- Dev commits change
- CI builds image and tags it
- CI pushes image to registry
Ops
- Pull image
- Stop/rm/rmi
- Tag?
- Run
docker pull, docker stop, docker rm/rmi, docker tag, docker run
multiple containers on different ports, HAproxy load balancing (https://github.com/DevTable/gantryd)
Key infrastructure
-
Image registry
-
Orchestration layer
Image registry
- Your own private registry → totally doable
- Registry Docker image available (of course)
- Supports different storage backends (files, S3, …)
setup:
# curl -sSL https://get.docker.com/ | sh
# Set DOCKER_OPTS="--insecure-registry example.com:5001" in /etc/default/docker
# sudo service docker restart
# registry only: gen SSL certs as per https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-14-04
REGISTRY $ docker run -d -p 5001:5000 --restart=always --name registry -v $PWD/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/dev-docker-registry.com.crt -e REGISTRY_HTTP_TLS_KEY=/certs/dev-docker-registry.com.key registry:2
REGISTRY $ docker ps
REGISTRY CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
REGISTRY c76096a64cab registry:2 "/bin/registry /etc/d" 3 seconds ago Up 2 seconds 0.0.0.0:5001->5000/tcp registry
REGISTRY $ docker images
REGISTRY REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
REGISTRY registry 2 2f1ef7702586 2 days ago 220.6 MB
REGISTRY $ docker images example.com:5001
REGISTRY REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
REGISTRY $ curl -k -i https://localhost:5001/v2/
REGISTRY HTTP/1.1 200 OK
REGISTRY Content-Length: 2
REGISTRY Content-Type: application/json; charset=utf-8
REGISTRY Docker-Distribution-Api-Version: registry/2.0
REGISTRY Date: Sat, 15 Aug 2015 23:54:13 GMT
REGISTRY
REGISTRY {}
REGISTRY
BUILDER $ docker pull busybox
BUILDER $ docker tag busybox example.com:5001/busybox:x.y.z
BUILDER $ docker push example.com:5001/busybox:x.y.z
RUNNER $ docker pull example.com:5001/busybox:x.y.z
RUNNER $ docker run -it example.com:5001/busybox:x.y.z
Orchestration
-
Deployment
-
Scaling and packing
-
Availability
Deployment: staging/prod, A/B testing, rolling updates
Scaling/packing: auto-scaling + effective use of hardware
Availability: auto-restart, load balancing
Orchestration
-
For small systems, DIY can work
Orchestration: zero downtime deploys and rollbacks with puppet.
Swarm beta: constraints, resource affinity.
placement strategies - bin packing, random, spread
constraints - based on labels you specify on daemon startup
affinities - automatic for -link, -volumes-from, -net=container
TODO: fault tolerance, HA scheduler
Kubernetes (koo-ber-net-ees or k8s):
- pods containing 1 or more cooperating containers
- Pods serve as units of deployment and horizontal scaling/replication.
- replication controllers create multiple pod replicas and create
replacements when one (or its host) fails
- no affinity-based co-scheduling
- Kubernetes encourages a flat address space and does not dynamically
allocate ports, instead allowing users to select whichever ports
are convenient for them. To achieve this, it allocates an IP address
for each pod.
- Service abstraction which provides a stable IP address and DNS name for
dynamic set of pods e.g. micro-service. Defined using label selector - can
refer to any set of pods. Connections forwarded by a local agent (kube proxy)
running on the source machine, to one of the back-end containers. Exact
back-end is chosen by round-robin. Kube proxy takes care of tracking pods as
they're replaced, so the service IP address (and DNS name) never changes.
- http://kubernetes.io/v1.0/docs/design/architecture.png?raw=true
- Turn Docker iptables networking off with --iptables=false --ip-masq=false
on the Docker dameon, but it should not trigger anyway.
- Public IP - real-deal IP of a host running a kube proxy. Portal IP is just
for inter-container access within kubernetes - portal IPs never leave a single
host.
- By default, the container hits its default gateway (the host) when looking
for the portal IP. The portal IP is referenced in the iptables rule which
sends that traffic to the host IP on a particular port. The kube-proxy
(running locally) is listening on that port, picks up that traffic, and
then load balances it round robin to a pod IP address. That pod can be
running on any other host across the kubernetes cluster.
- replication controllers can also be used to maintain availability of
master-elected, sharded, and worker-pool applications. Such applications
should use dynamic work assignment mechanisms, such as the etcd lock
module or RabbitMQ work queues, as opposed to static/one-time customization
of the configuration of each pod, which is considered an anti-pattern
- can change scaling programmatically, e.g. in response to monitoring feedback
- kubectl resize/rolling-update (creates a new replication controller)
Single process - oversimplification. Think single role instead.
Logging
Can output in logstash format (or text or json)
Real world Docker
@jlabusch
