Kartar.Net
If I had my hand full of truth, I would take good care how I opened it
TL;DR - Please take the 2015 Monitoring Survey
Last year I ran a monitoring survey, whose data I also reviewed as a series of posts on this blog and presented in several talks. I was interested in running the survey because I think we’re seeing the beginnings of a significant change in the maturity of the monitoring landscape.
I’ve decided to make the survey a yearly event and am coinciding the launch of this year’s survey with Monitorama in Portland.
Forthcoming book - The Art of Monitoring
One of the classic problems of monitoring alerts is that they are often very cryptic. Coupled with the challenge of alert fatigue1 this makes working out what to do next when you receive an alert quite tricky. Additionally, alerts often happen when we’re not at the top of our game: a 4am on a Sunday morning alert is not likely to foster an exemplary response.
One of my pet hates is having to maintain configuration inside monitoring tools. Not only large pieces like host definitions but smaller pieces like service and component definitions. Using a configuration management tool makes this much easier but it still generally requires some convergence to update your monitoring configuration when a host is added or removed or a service changes.
An example might be HAProxy. I have a HAProxy running with multiple back-end nodes.
TL;DR - This is not a comprehensive guide to Clojure, but it is enough to get you started with Riemann. This is also an excerpt from my forthcoming book - The Art of Monitoring. It'll also be available in the Riemann documentation at some point too.
Riemann is configured using a Clojure-based configuration file. This means your configuration file is actually processed as a Clojure program. So to process events and send alerts and metrics you'll be writing Clojure.
I’ve recently started alerting on expired events from Riemann via email. The default email alert looks something like this:
It contains some useful information but it is pretty basic: the subject is the name of the alerted service and the body contains a basic printout of the event’s fields.
I decided I’d like to build some alternative emails and so I went digging into the mailer plug-in code to find out how.