Bare Metal Chef

How to automate your hardware

by Maxime Brugidou / @brugidou

Clouds

knife ec2 server create

Prerequisites

• Network
• Firmwares
• BMC/IPMI

Chicken-Egg Problems

How do we install the first server?

Chef Server

DHCP

# Generated by Chef with DHCP cookbook
subnet 10.10.0.0 netmask 255.255.255.0 {
  next-server 10.10.0.3;
  option domain-name-servers 10.10.1.1, 10.10.1.2;
  option routers 10.10.0.1;
  range 10.10.0.128 10.10.0.255;
}
                    

Initial Diskless Image

Built with Debirf. Modified a bit to support:

• CentOS images using Mock
• rootmemsize kernel arg to limit the Diskless memory to 2GB
• firstboot_script kernel arg to run script after boot

PXE Boot

# TFTP cookbook
default['tftp']['pxelinux.cfg']['default']['netboot'] = {
  'kernel' => node['tftp']['files']['netboot_kernel']['name'],
  'append' => "initrd=#{node['tftp']['files']['initrd']['name']} 
  rootmemsize=2097152k console=tty0 console=ttyS1,9600n8 
  firstboot_script=tftp://#{node[:ipaddress]}/default/setup.sh"
}
					

Default Hostname

MAC Addresses are unique identifiers and deterministic. Example: 6c-3b-e5-a8-0f-34.test.org

cat <(cat /sys/class/net/*/address) \
<(cat /proc/net/bonding/* 2>/dev/null |\
    /bin/grep "Permanent HW addr:" |\
    /bin/awk '{print $4}') |\
/bin/grep -v 00:00:00:00:00:00 |\
/bin/sort | /usr/bin/uniq  |\
/usr/bin/head -n1 | /usr/bin/tr [A-Z] [a-z] |\
/usr/bin/tr : -
                   

Firstboot Script

• Install Chef
• Register Node
• Set run_list to role[firstboot]

Discover New Hardware

$ knife status 'roles:firstboot'
33 minutes ago, 2c-59-e5-3b-60-c4.test.org, 2c-59-e5-3b-60-c4.test.org, 10.12.160.74, centos 6.4.
31 minutes ago, 2c-59-e5-47-84-0c.test.org, 2c-59-e5-47-84-0c.test.org, 10.110.160.40, centos 6.4.
30 minutes ago, 74-46-a0-f4-1e-0c.test.org, 74-46-a0-f4-1e-0c.test.org, 10.110.160.54, centos 6.4.
30 minutes ago, f0-92-1c-10-17-04.test.org, f0-92-1c-10-17-04.test.org, 10.10.160.46, centos 6.4.
30 minutes ago, f0-92-1c-10-5a-b4.test.org, f0-92-1c-10-5a-b4.test.org, 10.12.160.79, centos 6.4.
...
                    

Provision a Node

Just give it a run_list!

RAID Controller

# raid cookbook
default[:raid][:config][:arrays] = [
  {
    :raid => "1+0",
    :disk_count => 8,
    :disk_size => "600 G",
    :disk_type => "SAS"
  },
  {
    :raid => "1"
    :disk_count => 2,
    :disk_size => "200 G",
    :disk_type => "Solid State SATA"
  }
]
                    

Disk Partitioning

# os cookbook
# fai-setup-storage syntax
default['os_install']['disks']['/dev/sda']['options'] = 'fstabkey:label disklabel:gpt-bios bootable:1'
default['os_install']['disks']['/dev/sda']['volume'] = [
  {
    :type => 'primary',
    :mountpoint => "/mnt/os_install",
    :size => '20GB-',
    :filesystem => 'ext4',
    :mount_options => 'defaults',
    :fs_options => "createopts=\"-L ROOT -b 4096 -m 5 -O extent\""
  },
  ...
]
                    

FAI Setup-Storage

• Part of the Debian FAI Project
• Works from our CentOS diskless image
• Can do anything (LVM, mdadm, ntfs...)
• We use it to partition and format on all platforms

OS On-Disk Installation

# os cookbook
default['os_install']['diskless'] = false
# supports: debian, centos, windows
default['os_install']['platform'] = 'centos'
                    

OS cookbook installation recipe

Other Tips

• Manage your BMC, it's vital
• Use dmidecode to get physical infos (It works on Windows too!)
• Use lldpd to discover your network neighbors, VLANs...
• Use ohai plugins to discover your hardware, location, etc...

Thanks! Questions?