Custom Nginx Metrics with Logster

Assumed Knowledge

Metrics, Graphite, Log Files, Some Linux, Some Python

The Problem

You want to make pretty graphs that let you measure things to do with third party applications.

...More Specifically

You want to measure the response codes nginx is firing out

The Solution

Fork the nginx repo and fire up gcc

The Solution

Logster - https://github.com/etsy/logster

Logster

Reads log files and turns them into metrics

Written in Python with metric parsers also written in Python

Intended to be invoked by cron or similar

Logster Parsers

10.41.78.12 - - [28/Aug/2015:08:02:22 +0000] "GET /healthcheck HTTP/1.1" 200 6 "-" "ELB-HealthChecker/1.0" 0.000 - .

Logster Parsers

class DemoLogster(LogsterParser):
    def __init__(self, option_string=None):
        self.http_404 = 0
		self.reg = re.compile('.*HTTP/1.\d\" (?P<http_status_code>\d{3}) .*')


    def parse_line(self, line):
        try:
            regMatch = self.reg.match(line)

            if regMatch:
                linebits = regMatch.groupdict()
                status = int(linebits['http_status_code'])
				if status == "404":
						self.404 = self.404 + 1
            else:
                raise LogsterParsingException("regmatch failed to match")

        except Exception as e:
            raise LogsterParsingException("regmatch or contents failed with %s" % e)


    def get_state(self, duration):
        self.duration = float(duration)

        return [
            MetricObject("http_404",
			(self.http_404 / self.duration),
			"Responses per sec")]

Now What?

Install Logster + Custom Scrapers

Setup a cron task to run once a minute and fire up Logster

Does This Actually Work?

Yes! We're using it for our brand new in production Lomo image resizing service

What About Other Applications?!

This can work with any log file you know the format of!

Thanks!

I'm Chris McCluskey from Travel Content Platform Squad