Catalyst

Introduction to Puppet

(1 day)

Presented by Evan Giles / Alex Lawn

Administrivia

• Bathrooms.
• Kitchen (water).
• Fire exits.
• Please ask questions as they occur to you (we can always take them offline if they are too tangential).

Introductions

• Who am I?
• Who are you?
• Expectations

Labwork Prep

• You can view these slides by pointing your web browser at the following URL:

http://catalyst-training.github.io/puppet-intro

• Load that up now, so you can refer to them as needed
• Start VirtualBox, start the three guests:
  • "puppet"
  • "node-a"
  • "node-b"
• Login (root/password)

Puppet is...

• A configuration management tool.
• Open Source (GPL).
• Written in Ruby
• Supported by a very active community both locally and around the world.
• Available for numerous Operating Systems.
• As simple or complex as you choose to make it.

Puppet is not...

• A programming language. It is idempotent.
• An orchestration tool.
• An intrusion detection system.

Why?

• Consistency.
• Auditabilty.
• Repeatability.

Leads to:

• Cost reduction.
• Reliability.

How?

• Centralisation of configuration.
  • Version control.
  • Ongoing changes
• Provider abstraction.

Installing Puppet

Usually you would use the puppet-release package from the PuppetLabs website. This installs the PuppetLabs Yum or apt repositories. https://docs.puppetlabs.com/guides/install_puppet/install_el.html

puppet# rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
puppet# yum install puppet-server

node-a# rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
node-a# yum install puppet

node-b# rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
node-b# yum install puppet

Start the puppetmaster.

• The puppetmaster has a built in web-server and listens on port TCP/8140.
• Production deployment should be within Apache/mod_passenger or similar.

puppet# systemctl enable puppetmaster
puppet# systemctl start puppetmaster
puppet# iptables --flush

Configure a puppet client.

• Puppet uses PKI authentication.
• All communication is SSL encrypted.
• By default the puppetmaster stands up it's own CA.
• Clients attempt to connect to the hostname 'puppet' unless configured otherwise.

Node A

On node-a, run the agent and instruct it to wait for the master to sign it's cert:

node-a# puppet agent -tv --waitforcert=120

List pending signing requests:

puppet# puppet cert list

Sign the client certificate:

puppet# puppet cert sign node-a.localdomain

Wait for the 120s to timeout

Let's test...

node-a# puppet agent -tv
Info: Caching certificate for node-a.localdomain
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for node-a.localdomain
Info: Retrieving plugin
Info: Caching catalog for node-a.localdomain
Info: Applying configuration version '1395248941'
Info: Creating state file /var/lib/puppet/state/state.yaml
Notice: Finished catalog run in 0.01 seconds

Slightly different way:

On node-b, run the agent but do not wait

node-b# puppet agent -tv

List pending signing requests:

puppet# puppet cert list

Sign all pending certs:

puppet# puppet cert sign --all

Run again on the agent

node-b# puppet agent -tv

Basic Manifests

Standalone

Let's make a user account... save this as user.pp on node-a

user { 'someuser':
  ensure => present,
  system => false,
  shell  => '/usr/bin/bash',
}

node-a# puppet apply user.pp

Modules

Let's install Apache...

puppet# cd /etc/puppet/modules
puppet# mkdir -p httpd/{manifests,templates,files}

modules/httpd/manifests/init.pp:

class httpd {
  package { 'httpd':
    ensure => 'installed',
  }
}

• Each pp file should contain a single class.
• The init.pp file must contain a class that matches the module name.

Locating the Puppet documentation (resource types).

• http://docs.puppetlabs.com/references/stable/type.html
• Lab offline copy available at the URL below:

http://PRESENTER_IP/typeref/references/stable/type.html

Assigning manifests to nodes

/etc/puppet/manifests/site.pp:

import 'nodes.pp'

/etc/puppet/manifests/nodes.pp:

node 'node-a.localdomain' {
  include httpd
}

Previewing changes.

node-a# puppet agent -tv --noop

Lab task

• Create a new puppet module called utils, apply to node-a.
• Install the utility iptraf.

• Check that the installation was successful by running iptraf.

  node-a# iptraf
  

• Configure puppet to REMOVE the package iptraf.

• Manually install iptraf.

  node-a# yum install iptraf
  

• Re-run puppet. What happened?

Creating a user.

Update the httpd manifest:

package { 'httpd':
  . . .
}
user { 'www':
  ensure     => present,
  managehome => true,
}

Did it work?

node-a# id www
node-a# ls -l /home

Grouping: A note on node inheritance

/etc/puppet/manifests/nodes.pp:

node 'webserver' {
  include httpd
}

node 'node-a.localdomain' inherits webserver {
  include utils
}

node 'node-b.localdomain' inherits webserver {
}

Up until puppet 3.7, you can do this, but you shouldn't:

From 3.7, it's deprecated, and going away in 4.0. Use with caution

Preferred, inheritance:

modules/roles/manifests/webserver.pp:

class roles::webserver {
  include httpd
}

modules/roles/manifests/specialwebserver.pp:

class roles::specialwebserver {
  include roles::webserver
  include utils
}

manifests/nodes.pp:

node 'node-a.localdomain' {
  include roles::specialwebserver
}

node 'node-b.localdomain' {
  include roles::webserver
}

Or even better, composition:

modules/roles/manifests/webserver.pp:

class roles::webserver {
  include httpd
}

modules/roles/manifests/specialwebserver.pp:

class roles::specialwebserver {
  include utils
}

manifests/nodes.pp:

node 'node-a.localdomain' {
  include roles::specialwebserver
  include roles::webserver
}

node 'node-b.localdomain' {
  include roles::webserver
}

Pushing a file.

modules/httpd/manifests/init.pp:

file { '/var/www/html/index.html':
  ensure => 'present',
  owner  => 'www',
  group  => 'apache',
  mode   => '0444',
  source => 'puppet:///modules/httpd/var/www/html/index.html',
}

Where did it come from?

It is more manageable to create a directory structure within puppet that matches the filesystem layout; however this is not enforced by puppet:

puppet# mkdir -p /etc/puppet/modules/httpd/files/var/www/html

modules/httpd/files/var/www/html/index.html:

<html>
  <head>
    <title>Test Page</title>
  </head>
  <body>
    <h1>Hello World!</h1>
  </body>
</html>

curl http://localhost

Managing a service.

modules/httpd/manifests/init.pp:

service { 'httpd':
  enable => true,
  ensure => running,
}

Run puppet

Stop httpd service

Run puppet again

curl http://localhost

Dependencies and Ordering

node-a# systemctl stop httpd
node-a# userdel www
node-a# rpm -e httpd
node-a# rm -f /var/www/html/index.html

Run puppet

Dependencies and Ordering

service { 'httpd':
  enable  => true,
  ensure  => 'running',
  require => Package['httpd'],
}

• note the capital letter and square brackets!

node-a# rpm -e httpd
node-a# puppet agent -tv

Dependencies and Ordering

More comprehensive require:

service { 'httpd':
  enable  => true,
  ensure  => 'running',
  require => [ Package['httpd'], File['/var/www/html/index.html'] ],
}

Pedantic and not strictly necessary, just an example

Dependencies and Ordering

Happiness is a node that self-builds with puppet in one run.

Lab task

• Bring the current /etc/httpd/conf/httpd.conf under Puppet control.
• Change the Apache port to 8080 (Listen 8080).

Is Apache listening on port 8080 now?

node-a# netstat -anp | grep -w LISTEN
node-a# systemctl restart httpd
node-a# netstat -anp | grep -w LISTEN

• This is less than ideal, we have to restart by hand.

Subscribe/Notify

Two options - don't do both!

service { 'httpd':
  enable    => true,
  ensure    => 'running',
  require   => [ Package['httpd'], File['/var/www/index.html'] ],
  subscribe => File['/etc/httpd/conf/httpd.conf'],
}

file { '/etc/httpd/conf/httpd.conf':
  ensure  => present,
  owner   => 'root',
  group   => 'root',
  mode    => '0644',
  source  => 'puppet:///modules/httpd/etc/httpd/conf/httpd.conf',
  require => Package['httpd'],
  notify  => Service['httpd'],
}

Change port to 8081, run puppet

Before

Before can be used instead of require, on the resource that needs to go first.

Similar to subscribe vs notify.

Generally prefer notify and require; tends to be clearer

Chaining

Generally replaces require/subscribe/notify, but can coexist with care.

package { 'httpd':
  . . .
} ->
user { 'www':
  . . .
} ->
file { '/var/www/index.html':
  . . .
} ->
file { '/etc/httpd/conf/httpd.conf':
  . . .
} ~>
service { 'httpd':
  . . .
}

Another way of Chaining

Package['httpd'] -> User['www'] -> File['/var/www/index.html'] ->
File['/etc/httpd/conf/httpd.conf'] ~> Service['httpd']

Dynamicism // Templates.

file { '/var/www/index.html':
  ensure  => present,
  owner   => 'www',
  group   => 'apache',
  mode    => '0444',
  content => template('httpd/index.html.erb'),
}

modules/httpd/templates/index.html.erb

<html>
  <head>
    <title>Test Page</title>
  </head>
  <body>
    <h1>Hello World!</h1>
    <p>I am running on <%= @hostname %>.</p>
  </body>
</html>

Facter

• We just used a pre-defined dynamic variable from facter.

  node-a# facter
  node-a# facter hostname
  

• Custom facts possible
  • Written in Ruby
  • Can be anything

Hiera

• discussion and demo

Idempotency of execution.

• Discussion, especially --noop.

file { '/etc/aliases' :
  . . .
}

exec { 'newaliases' :
  command     => '/usr/bin/newaliases',
  refreshonly => true,
  require     => File['/etc/aliases'],
  subscribe   => File['/etc/aliases'],
}

exec { 'ensure-no-anonymous-user':
  onlyif  => '/usr/bin/mysql -u \"\" -e \"show status\"',
  command => '/usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf -e \"drop user ''@'localhost'; drop user ''@'$hostname'\"',
  require => Service['mysql'] ,
}

Re-deployments.

Simulate a host-rebuild:

node-a# systemctl stop puppet
node-a# rm -rf /var/lib/puppet/ssl
node-a# puppet agent -tv

Re-deployments.

On the master:

puppet# puppet cert clean <name>

On the client:

node-a# systemctl stop puppet
node-a# rm -rf /var/lib/puppet/ssl

Then re-sign as though fresh

Lab task

• Rename node-a to node-c and reconnect to Puppet.

/etc/sysconfig/network
/etc/hosts
---
/etc/puppet/manifests/nodes.pp

Making it Better

Formatting conventions.

• Styleguide: https://docs.puppetlabs.com/guides/style_guide.html
• Always use trailing commas!
• puppet-lint (commit hooks)
• Ordering of attributes in types: consistency is key

Using Puppet Forge.

http://forge.puppetlabs.com/

puppet module install <module>

• puppetlabs-stdlib is ultra handy!

Extras - Conditionals

if

if $hostname == 'node-a' {
  $owner = 'root'
} else {
  $owner = 'www'
}

file { '/var/html/index.html':
  . . .
  owner => $owner,
  . . .
}

Selectors

$owner = $hostname ? {
  'node-a'  => 'root',
  'node-b'  => 'www',
  default   => 'www-data',
}

file { '/var/html/index.html':
  . . .
  owner => $owner,
  . . .
}

Template Logic

index.html.erb:

<html>
  <head>
    <title>Test Page - <%= owner %></title>
  </head>
  <body>
    <h1>Hello World!</h1>
    <p>I am running on <%= hostname %>.</p>
    <% if memorysize < '1024' %>
      <p><em>This is a low memory node.</em></p>
    <%end %>
  </body>
</html>

Extras - Multi-source file

Sourceselect

Using sourceselect, we can merge directory structures on the puppetmaster into per-host specific combinations

file { '/etc/sudoers.d/':
  source => ["puppet:///modules/sudo/etc/sudoers.d.${hostname}",
           "puppet:///modules/sudo/etc/sudoers.d"],
  sourceselect => all,
  recurse      => true,
}

Extras - Resource Defaults

Defaults

class test {
  file { '/var/lib/myapp/file1':
    source => "puppet:///modules/${module_name}/var/lib/myapp/file1",
    owner  => 'myappuser',
    group  => 'adm',
    mode   => '644',
  }
  file { '/etc/myapp.cfg':
    source => "puppet:///modules/${module_name}/etc/myapp.cfg",
    owner  => 'myappuser',
    group  => 'root',
    mode   => '644',
  }
  file { '/etc/myapp.d/someconfig':
    source => "puppet:///modules/${module_name}/etc/myapp.d/someconfig",
    owner  => 'myappuser',
    group  => 'root',
    mode   => '644',
  }
}

Tedious, no?

Much simpler

class test {
  File {
    owner  => 'myappuser',
    group  => 'root',
    mode   => '644',
  }
  file { '/var/lib/myapp/file1':
    source => "puppet:///modules/${module_name}/var/lib/myapp/file1",
    group  => 'adm',
  }
  file { '/etc/myapp.cfg':
    source => "puppet:///modules/${module_name}/etc/myapp.cfg",
  }
  file { '/etc/myapp.d/someconfig':
    source => "puppet:///modules/${module_name}/etc/myapp.d/someconfig",
  }
}

Applies only to the current class.

Can include things like "require" and "notify"

Extras - Tidbits

Local testing

Create a modules dir, then a "foo" module in that dir. Run it locally with:

puppet apply --modulepath=~/modules -e "include foo"

--noop also possible

Saving bacon

• Syntax checking:

puppet parser validate $file

• Lint for style:

yum install rubygem-puppet-lint.noarch
puppet-lint $file

• In git pre-commit hooks on working copies for the win.