Automated Documentation of Monitoring Thresholds

View Github Repository Open presentation in a new window

mattyjones

See all presentation from mattyjones

Automated Documentation of Monitoring Thresholds

0 0

estark

[Slides] Automated Documentation Of Monitoring Thresholds

On Github mattyjones / estark

Automated Documentation of Monitoring Thresholds

Yes you need this

Problem

What are we are monitoring

  • 10k machines
  • 250k checks
  • worldwide user base

Users needed to know what the current thresholds were for their machines

Tool Belt

Icinga 1x (Application)

fork of Nagiostext file based configurarionone file per server

Gitlab (Single Source of Truth)

Open Source eq. of GithubLDAP based permissionsrenders all .md files

Markdown (Markup Language)

easy to writeeasy to rendercan be treated as code

Requirements

easy to findeasy to updateeasy to readaudit trail

Solution

Bob opens a ticket to change the monitoring on foo-01.prd.aws.com monitoring updates the configuration file foo-01.prd.md the commit msg contains the ticket id Icinga has a check polling its configuration repo when a new commit is made pull the latest changes and apply bash fu monitoring updates the ticket with the commit id and link to the config file

End Result

https://gitlab.hal2k.com/aws/ldap/foo-01.prd.aws.com.md

Twitter: @devopsmatt Github: mattyjones Linkedin: Matty Jones Yieldbot is hiring [X] Slides: mattjones-pres-monitorama2015.herokuapp.com

Automated Documentation of Monitoring Thresholds Yes you need this