puppet-intermediate

View Github Repository Open presentation in a new window

catalyst-training

See all presentation from catalyst-training

puppet-intermediate

0 0

puppet-intermediate

Puppet intermediate training slides

On Github catalyst-training / puppet-intermediate

Catalyst

Intermediate Puppet

(2 days)

Presented by Evan Giles / Alex Lawn

Administrivia

  • Bathrooms.
  • Kitchen (water).
  • Fire exits.
  • Please ask questions as they occur to you (we can always take them offline if they are too tangential).

Pre-requisites

  • Download the Open Virtualisation Format lab image:
     http://PRESENTER_IP/files/puppet_int_lab_nodes.ova
  • Import the OVA into VirtualBox. Make sure you do NOT reinitialise the MAC addresses of the network interfaces.

These slides

You can view these slides at http://catalyst-training.github.io/puppet-intermediate/

There'll be quite a lot of copy/pasting coming up, so take the time to load them up now...

Basic re-cap

Installing Puppet

puppet# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
puppet# yum install puppet-server

node-a# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
node-a# yum install puppet

node-b# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
node-b# yum install puppet

node-d# wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
node-d# dpkg -i puppetlabs-release-trusty.deb
node-d# apt-get update
node-d# apt-get install puppet
node-d# service puppet stop

Start the puppetmaster / sign the clients

puppet# chkconfig puppetmaster on
puppet# /etc/init.d/puppetmaster start

Repeat for all nodes:

node-x# puppet agent -tv --waitforcert=300
puppet# puppet cert --list
puppet# puppet cert --sign node-x.localdomain
node-x# puppet agent -tv

Classify the clients

/etc/puppet/manifests/site.pp:

import 'nodes.pp'

/etc/puppet/manifests/nodes.pp:

node 'node-a.localdomain' {
}
node 'node-b.localdomain' {
}
node 'node-d.localdomain' {
}

File locations

  • /etc/puppet.conf
  • /etc/puppet/manifests
  • /etc/puppet/modules/<modulename>/manifests
  • /etc/puppet/modules/<modulename>/templates
  • /etc/puppet/modules/<modulename>/files

Create a module that:

  • Is called web.
  • Installs apache on node-a/b ONLY.
  • Creates a user "web".
  • Manages the apache httpd.conf file.
  • Pushes the index.html file below (owned by the web user).
    <html>
  <head>
    <title>Test Page</title>
  </head>
  <body>
    <h1>Hello World!</h1>
  </body>
</html>

Check you answer

package { 'httpd':
  ensure => 'installed',
}

user { 'web':
  ensure => present,
  managehome => true,
}

file { "/var/www/index.html":
  ensure => present,
  owner => "www",
  group => "apache",
  mode => "444",
  source => "puppet:///modules/web/var/www/index.html",
  require => [ User["web"], Package['httpd'], ],
}

service { 'httpd':
  enable => true,
  ensure => 'running',
  require => [ Package["httpd"], File["/etc/httpd/conf/httpd.conf"] ],
}

file { "/etc/httpd/conf/httpd.conf":
  ensure => present,
  owner => "www",
  group => "apache",
  mode => "444",
  source => "puppet:///modules/web/etc/httpd/conf/httpd.conf",
  require => User["web"],
  notify => Service["httpd"],
}

But what about node-d

if $osfamily == 'RedHat' {
  $http_pkg = 'httpd'
} elsif $osfamily == 'Debian' {
  $http_pkg = 'apache2'
} else {
  warning( 'OS not supported.' )
}
  • Update your manifest to use the $http_pkg variable.

Is there a more elegant solution?

$http_pkg = $osfamily ? {
  'RedHat'  => 'httpd',
  'Debian'  => 'apache2',
}

Test...

notify { 'Whassup my homies.': }

Could this be more sensible...

notify { "http_pkg is $http_pkg": }

What about the config file?

source => [
  "puppet:///modules/web/etc/httpd/conf/httpd.conf-$osfamily",
  "puppet:///modules/web/etc/httpd/conf/httpd.conf",
],

We must ensure that one option ALWAYS matches otherwise the catalogue compilation will fail.

We also have unless:

unless $fqdn == 'node-d.localdomain' {
  package { 'tree':
    ensure => 'installed',
  }
}

Help (Docs / RAL)

  • Google: puppet types
  • puppet describe user
  • puppet resource file /etc/passwd
  • puppet resource service
  • puppet resource package
- RAL == Resource Abstraction Layer - ...allows puppet to define existing resources...

Environments

  • Use directory based environments. All other options deprecated.
  • Scenario: production should listen on port 80, dev should listen on port 8080.

Configuring environments

/etc/puppet/puppet.conf

[master]
  environmentpath = $confdir/environments
  • Puppet will now ignore the manifest, modulepath, and config_version directives in puppet.conf.
  • Puppet will also ignore any environment blocks in puppet.conf. - This setting is on the puppet master
  • environment blocks were the old way of doing it

A new directory structure:

mkdir /etc/puppet/environments/{production,dev}
mv /etc/puppet/manifests /etc/puppet/environments/production/
mv /etc/puppet/modules /etc/puppet/environments/production/
- This 'production' is the default...

Place node-b in the dev environment: /etc/puppet/puppet.conf (on node-b)

  [agent]
    environment = dev
  • How should we apply the web class to node-b?- we need to update the files in the 'dev' environment...
  • Configure dev to listen on port 8080 and test.
  • Our dev and production environments have now diverged.
  • This is not so cool, because we will have trouble migrating changes from one environment to the next...

Let's abstract the port logic.

$http_port = $environment ? {
  'production' => '80',
  'dev'        => '8080',
}

content => template("httpd/httpd.conf.erb"),

Listen <%= @http_port %>

Namespace

  • Let's break the code into manageable chunks:
    class web {
. . .
}
class web::infrastructure {
. . .
}
class web::application {
. . .
}

Include our new classes

include web::infrastructure
include web::application

What about dependencies?

include web::infrastructure
class { 'web::application':
  require => Class['web::infrastructure'],
}

Does our template still work?

<%= scope['web::http_port'] %>

Tags

  • Enable us to selectively push elements of the catalogue.
  • Are implicitly created for each class.
  • Create a new module called note that prints a message with the notify resource type.
  • Make a change in the web class.
node-a# puppet-agent -tv --tags web
node-a# puppet-agent -tv --tags note

Explicit

  • Assign a tag to a class and all of the resources in contains.
    class 'web::application' {
tag 'app'
}

node-a# puppet-agent -tv --tags app

Functions

  • Google "puppet functions".
  • We've already seen "template".
  • Let's look at fqdn_rand.
  • Returns a "random" number based on the hash of the fqdn. Will never change for THAT host.
  • Good for spreading cron jobs across nodes.
$hour = fqdn_rand( max - min ) + min

Lab task

  • Use the above logic to create a cron entry that executes every x minutes where x is between 1 and 10.
  • Your job should call "/bin/false". You can watch /var/log/cron to see if it worked.

Template logic

  • Anything that Ruby ERB supports.
  • Everything in Ruby is an object.
  • Think object.method in all cases.

Conditionals

<% if @osfamily == 'Debian' %>
  I'm a bit different to the others.
<% end %>

Annoying whitespace

(Because we are diligent at indenting.)

<%- if @osfamily == 'Debian' %>
  I'm a bit different to the others.
<% end %>

Loops

$colours = [ 'red', 'yellow', 'pink', 'green', 'purple', 'orange', 'blue', ]

I can sing a rainbow:
<% @colours.each do |c| %>
  Colour: <%= c %>
<% end %>

Annoying whitespace

I can sing a rainbow:

<% @colours.each do |c| -%>
  Colour: <%= c %>
<% end -%>

Ruby

<%
  # Establish an empty array.
  @cmd = []

  # Push a string onto the array.
  @cmd << 'mk.rainbow'

  # Add my commandline parameters for colours (interpolate quotes).
  @colours.each do |c|
    cmd << --include
    cmd << "'#{c}'"
  end
-%>
<%= @cmd.join(' ') %>

Syntax validation

erb -P -x -T '-' mytemplate.erb | ruby -c

Custom facts

environments/XXX/modules/web/lib/facter/pkg_count.rb

Facter.add( 'pkg_count' ) do
  setcode 'rpm -qa | wc -l'
end
- This magically deploys the fact to the nodes - you need to run facter -p to see it

Confine by OS

environments/dev/web/lib/facter/pkg_count.rb

Facter.add( 'pkg_count' ) do
  confine :kernel => "Linux"
  setcode 'rpm -qa | wc -l'
end

What about Ubuntu?

environments/dev/web/lib/facter/pkg_count.rb

Facter.add( 'pkg_count' ) do
  confine :kernel => "Linux"
  setcode do
    osfamily = Facter.value(:osfamily)
    case osfamily
    when 'RedHat'
      Facter::Core::Resolution.exec('rpm -qa | wc -l')
    when 'Debian'
      Facter::Core::Resolution.exec('dpkg -l | wc -l')
    end
end

External facts

On the client node: /etc/facter/facts.d/lab_facts.txt

product_key=DIndiwnsklIndk
thingee=mawhatsit

Debugging

facter --debug

Parametized classes

class web::infrastructure (
  $http_port = '80',
  $server_name,
) {
  . . .
}
  • Rewrite your web manifest to use a parametized class.
  • node-a should listen on port 80, 'node-b` on port 81.
  • The index.html should display the $servername variable.

How do we get those parameters in from our nodes.pp?

node 'node-a.localdomain' {
  class { web::infrastructure:
    server_name => "I am node A.",
  }
}
node 'node-a.localdomain' {
  class { web::infrastructure:
    server_name => "I am node B.",
  }
}

We are missing web::aplication:

node 'node-a.localdomain' {
  class { web::infrastructure:
    server_name => "I am node A.",
  }
  class { web::application:
    require => Class['web::infrastructure'],
  }
}
node 'node-a.localdomain' {
  class { web::infrastructure:
    server_name => "I am node B.",
  }
  class { web::application:
    require => Class['web::infrastructure'],
  }
}

Or we could parametize the web class as well:

class web (
  $http_port = '80',
  $server_name,
) {
  class { 'web::infrastructure':
    server_name => $server_name,
    http_port   => $http_port,
  }
  class { 'web::application':
    require => Class['web::infrastructure'],
  }
}

class web::infrastructure (
  $http_port = '80',
  $server_name,
) {
  . . .
}

Or we could use inheritance:

class web::infrastructure::params {
  $http_port = '80'
  $server_name = $::fqdn

  # Could include package selection logic here.
}

class web::infrastructure (
  $http_port   = $web::infrastructure::params::http_port,
  $server_name = $web::infrastructure::params::server_name,
) inherits web::infrastructure::params {
  . . .
}

node 'node-a.localdomain':
  class { 'web::infrastructure::params':
    server_name = "I am node A."
  }
  class { 'web': }
}

Define

  • Used for building our own resouce type.
  • Can be called more than once (unlike a class).
class dudes {
  define dude (
    $ensure,
    $dude_meter = true,
  ) {

    user { $name:
      ensure     => $ensure,
      managehome => true,
    }

    $ensure_dude_meter = $dude_meter ? {
      true  => 'present',
      false => 'absent',
    }

    file { "/home/$name/dude_level.txt":
      ensure  => $ensure_dude_meter,
      owner   => $name,
      group   => 'root',
      mode    => '444',
      content => fqdn_rand(10, $name),
    }

  }

  dude { 'dude1':
    ensure => 'present',
  }

  dude { 'dude2':
    ensure => 'present',
  }

  dude { 'dude3':
    ensure     => 'present',
    dude_meter => false,
  }

}

mod_passenger / apache

puppet# yum -y install httpd mod_ssl mod_passenger
puppet# chkconfig puppetmaster off
puppet# /etc/init.d/puppetmaster stop

puppet# mkdir -p /srv/rack/puppetmasterd/{tmp,public}
puppet# cp /usr/share/puppet/ext/rack/config.ru /srv/rack/puppetmasterd
puppet# chown -R puppet:puppet /srv/rack/puppetmasterd

/etc/httpd/conf.d/passenger.conf:

# And the passenger performance tuning settings:
PassengerHighPerformance On

# Set this to about 1.5 times the number of CPU cores in your master:
PassengerMaxPoolSize 12

# Recycle master processes after they service 1000 requests
PassengerMaxRequests 1000

# Stop processes if they sit idle for 10 minutes
PassengerPoolIdleTime 600

/etc/httpd/conf.d/puppetmasterd.conf:

Listen 8140
<VirtualHost *:8140>
  SSLEngine On

  SSLProtocol             All -SSLv2
  SSLCipherSuite          HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP
  SSLCertificateFile      /var/lib/puppet/ssl/certs/puppet.<domain>.pem
  SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/puppet.<domain>.pem
  SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
  SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
  SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
  SSLVerifyClient         optional
  SSLVerifyDepth          1
  SSLOptions              +StdEnvVars +ExportCertData

  # These request headers are used to pass the client certificate
  # authentication information on to the puppet master process.
  RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
  RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
  RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

  DocumentRoot /srv/rack/puppetmasterd/public

  <Directory /srv/rack/puppetmasterd/>
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

puppet# /etc/init.d/httpd start
puppet# chkconfig httpd on
node-a# puppet agent -tv

Foreman

  • Work in groups.
  • Make a copy of your modules.
  • Uninstall puppetmaster/apache/etc.
  • Install and run the foreman installer:
yum -y install http://yum.theforeman.org/releases/1.1/el6/x86_64/foreman-release.rpm
yum -y install foreman-installer
ruby /usr/share/foreman-installer/generate_answers.rb

Login

  • https://puppet.localdomain/
  • username: admin
  • password: changeme

open source technologists

Catalyst Intermediate Puppet (2 days) Presented by Evan Giles / Alex Lawn