• Basic and rapid introduction
• Coding the CI / CD infrastructure
• Coding the build - part 1 (Jenkins)
• Coding the build - part 2 (build in containers)
• Coding the build - part 3 (docker in docker)

Docker

Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries - anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.

Docker architecture

Docker Compose

Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Docker compose by example

slides:
  image: nginx

  environment:
    VIRTUAL_HOST: 'nextbuild.docker'
    VIRTUAL_PORT: 80
  volumes:
    - .:/usr/share/nginx/html
  ports:
    - "8889:80"

Build environment

Build environment

Code the infrastructure

docker-compose.yml

gitlab:
  image: 'gitlab/gitlab-ce:latest'
  hostname: 'gitlab.docker'
  ports:
    - '81:8080'
  volumes:
    - '/opt/gitlab/config:/etc/gitlab'
    - '/opt/gitlab/logs:/var/log/gitlab'
    - '/opt/gitlab/data:/var/opt/gitlab'

jenkins:
  image: 'jenkins:1.642.3'
  links:
    - gitlab:gitlab.docker
  ports:
    - '82:8080'
  volumes:
    - '/opt/jenkins/jenkins_home:/var/jenkins_home'

Code the infrastructure

OKAY now we have GIT and Jenkins running. But we still need to install some magick: the jenkins plugins.

Code the infrastructure

• Create a jenkins docker image on top of the jenkins image.
• Jenkins base image provides a mechanism to specify the plugins.

Code the infrastructure

MAINTAINER Niek Palm dev.npalm@gmail.com
FROM jenkins:1.642.3

COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt

workflow-support:1.15
workflow-multibranch:1.15
workflow-step-api:1.15
git:2.4.4
workflow-basic-steps:1.15
workflow-api:1.15
workflow-scm-step:1.15
pipeline-rest-api:1.0
pipeline-stage-view:1.0

Code the infrastructure

Update docker-compose.yml

gitlab:
  image: 'gitlab/gitlab-ce:latest'

  ...

jenkins:
  build: 'jenkins'
  links:
    - gitlab:gitlab.docker
  ports:
    - '82:8080'
  volumes:
    - '/opt/jenkins/jenkins_home:/var/jenkins_home'

Code the build

Code the build - Sample app

• The sample application is a Java based Spring Boot service witch uses a mongo db (data store).

• The build is scripted in gradle. We can build by running:

  gradlew <tasks>
  

• The application is distributed as a docker container.

Code the build - Jenkins Pipeline

• Jenkins pipeline plugin provides a way to code the build via a script file.
• Jenkins pipeline will be part of the new coming Jenkins 2.0
• Steps
  • Create a Jenkinsfile to code the build.
  • Create a pipeline job which uses SCM (GIT) to get the Jenkinsfile.

Code the build - Jenkins

Jenkinsfile

node {
  git url: 'http://gitlab.docker/nextbuild/service.git'

  stage 'Clean'
  sh './gradlew clean'

  stage 'Build'
  sh './gradlew assemble'

  stage 'Test'
  sh './gradlew check'

}

Jenkins build

Artifactory and Sonar

Managing artifacts - Artifactory

• Next we add Artifactory OSS as repository to maintain artifacts.
  • Add Artifactory OSS CI ecosystem and update docker-compose.
  • Gradle build is already aware of the repository.

Artifactory - Update Compose

artifactory:
  image: 'jfrog-docker-reg2.bintray.io/jfrog/artifactory-oss'
  ports:
    - '87:8081'
  volumes:
    - '/opt/artifactory/data:/var/opt/jfrog/artifactory/data'
    - '/opt/artifactory/logs:/var/opt/jfrog/artifactory/logs'
    - '/opt/artifactory/backup:/var/opt/jfrog/artifactory/backup'

Measuring Code Quality - Sonar

• Next we add SonarQube to measure the code qaulity.
  • Add SonarQube to our CI ecosystem and update docker-compose.
  • Add Sonar to the build and update Jenkinsfile.

Sonar - Update Compose

sonardb:
  image: postgres:9
  environment:
    POSTGRES_USER: 'sonar'
    POSTGRES_PASSWORD: 'secret'

sonar:
  image: sonarqube:5.4
  environment:
    SONARQUBE_JDBC_USERNAME: 'sonar'
    SONARQUBE_JDBC_PASSWORD: 'secret'
    SONARQUBE_JDBC_URL: 'jdbc:postgresql://sonardb.docker/sonar'
  links:
    - sonardb:sonardb.docker
    - gitlab:gitlab.docker
  ports:
    - "9000:9000"

jenkins:
  ...
  links:
    ...
    - sonardb:sonardb.docker
    ...

Sonar - Update Jenkinsfile

node {
  git url: 'http://gitlab.docker/nextbuild/service.git'

  ...

  stage 'QA'
  sh './gradlew sonarRunner'

}

Code the infrastructure part 2

• Running a build in a container provides a reproducable and consistent environment.
• GitLab CI fits better because of the model of runners that supports docker.

GitLab CI - Runners

Code the build - GitLab CI

• GitLab CI is fully integrated in GitLab.
• GitLab CI builds automatically once a .gitlab-ci.yml file is in the root of the repo.

.gitlab-ci.yml snippet

stages:
  - build

assemble:
  stage: build
  script:
    - ./gradlew clean assemble

Code the build in GitLab CI

We have GitLab CI already running since we use GitLab as our GIT server. So, we only need to create a runner.

gitlabrunner:
  image: 'gitlab/gitlab-runner:latest'
  environment:
    REGISTRATION_TOKEN: 'JCqWxz4LcNmth4F4PdTo'
    CI_SERVER_URL: 'http://gitlab.docker/ci'
  ...

> docker exec -i -t gitlab-runner-dind1 gitlab-runner register -n \
   --docker-links 'gitlab_gitlab_1:gitlab.docker'

Code the build in GitLab CI

.gitlab-ci.yml for our service

stages:
  - build
  - test
  - qa

assemble:
  stage: build

  image: npalm/java:oracle-java8

  script:
    - ./gradlew clean distTar

test:
  stage: test

  image: npalm/java:oracle-java8

  script:
    - ./gradlew check

Code the build in GitLab CI

.gitlab-ci.yml for our service

stages:
  - build
  - test
  - qa

assemble:
  ...

test:
  ...

sonar:
  stage: qa

  image: npalm/java:oracle-java8

  script:
    - ./gradlew sonarRunner

Code the build part 3

• How can we run end-to-end test for our service?

Test our app using docker

• Define a docker-compose file to start our service and dependencies.
  • The service has a Dockerfile which defines the service container.
  • For mongo we use the official docker image.

mongo:
  image: mongo:3.2
service:
  build: ./build/docker
  ports:
    - "8888:8080"
  links:
    - mongo:mongodb

Docker in Docker

Code the build part 3

.gitlab-ci.yml

stages:
  - build
  - test
  - qa

...

performance_test:
  stage: test

  image: npalm/dind-java:latest

  script:
    - ./gradlew assemble copyDockerfiles
    - docker-compose build
    - docker-compose up -d
    - ./wait.sh service_service_1 service 8888
    - ./gradlew jmeterRun

...