How to Write Runbooks People Can Actually Follow at 3am

We are around the corner of Black Friday and Holiday season, every SRE’s favorite season, because tensions are at an all-time low (wink wink) and Christmas Spirit is all around us.

Also, there is something else that tends to happen around this time - you guessed it - people are reviewing their runbooks! Hurray!

Runbooks are very peculiar artifacts in our industry, as they often suffer from extreme negligence and most are written without people pouring their hearts into the them, which makes them said and abandoned, never to be read.

Read more →

Implementing SLOs Without the Theatre

Every few months someone in a leadership meeting discovers SLOs (Service Level Objectives), and what follows is something along the following lines: A spreadsheet appears where every service gets a row. Every row gets 99.9% if “it’s important” and 99.5 if “it’s not critical” because that looks good, “we will adjust it later”. A dashboard is built, a Confluence page is written, and then nothing happens for about eighteen months until somebody asks why we are still getting paged at 3am for things nobody cares about.

Read more →