Learn how to set up Alerts & On-call in Compass in 5 easy steps

[#1]

Compass Operations gives teams context-rich alerts and on-call features alongside a complete software catalog. Pairing alerts with critical info like service ownership, dependencies, and scorecards makes it easier to respond to and resolve alerts, and streamlines workflows for engineering teams. Less time to respond to alerts means fewer downtimes and higher SLAs. Compass empowers development teams to maintain the highest levels of performance and reliability.

In this post, we’ll cover all the basic steps to enable Compass Operations for your team and begin receiving notifications from your monitoring tool of choice.

What is Compass Operations?

A few months ago, we announced the General Availability of the Alert and On-call capabilities in Compass. Along with the basic features of alerts and on-call, we also launched a whole host of supporting features that help you manage your incoming alerts better. We call the collection of these features “Operations.” These features enable you to proactively monitor alerts generated by your tech stack and help you resolve them quickly.

This post will help you understand how to set up your organization in your Compass Operations platform.

Five easy steps to get started

When you create a new instance of Compass, there are just five simple steps to start receiving alerts for your teams:

1. Create a Team in Compass
2. Enable Operations for your team
3. Review your on-call schedule
4. Create an Integration with your monitoring tool
5. Configure how you want to receive notifications

Let’s explore these steps one by one.

Step 1 - Create a Team in Compass

When you start with a new instance of Compass, the first action to take is to create a Team. Simply go to “Teams” in the top navigation and select “Create a Team”.

In this step, you can configure:

1. Team name

2. Team members

3. Permission to allow anyone on your Atlassian Site to join this team

Everything that you set at this stage can be edited later. After you click on “Create a Team”, your team gets created.

Step 2 - Enable Operations for your Team

This is an important step to unlock the Alert and On-call capabilities for your team. Without performing this step, you can't set on-call schedules for your teams or receive alerts.

This simple process can be performed from either of two places: 

Option 1: Team → Operations

Option 2: Operations → Overview

Regardless of where you start your journey from, you will be presented with the following experience

At this step, you can:

1. Assign specific users as Admins of the Team

2. Assign all users as Admins of the Team

These Team Admins can manage configurations related to alerts and on-call for this team. You can always change who is a Team admin at any later point.

This two-step process enables Operations for your team, and you are now ready to start exploring Operations

Step 3 - Review your on-call schedule

An on-call schedule answers the question, “Who is responsible for responding to alerts for a given team?” The Compass on-call schedule allows you to easily set up a rotating roster of your team members so that each member can rotate through their designated on-call responsibilities. By default, Compass creates a sample on-call schedule, a routing rule, and an escalation policy for each team that has Operations enabled.

Expanding on these concepts a bit more:

• Routing rules: When an Alert arrives for a Team, this is the first stop in determining whom to page for the alert. The value entered in the routing rule can be either a schedule or an escalation policy. You can have different routing rules catering to different types of alerts. For example, you can choose to route all your P3 and P2 alerts to your junior on-call responders, and route all your P1 and P0 alerts to your team of senior developers
  (* Alerts are usually classified on priorities which are designated by P0, P1 etc. The lower the numeral, the higher the priority)

• Escalation Policy: Along with the Routing Rules, you can also decide how your alerts get handled using the escalation policies. This is especially helpful if the objective of your organisation to minimise downtimes and maintain SLAs. An escalation policy helps you to automatically escalate any alert up one level should it not get resolved or acknowledged in time. For example, you can define a policy where all alerts by default get sent to your level 1 on-call team. However, should an alert not get acknowledged in 10 mins, you can automatically have the alert sent to the level 2 on-call team or the next person in your on-call schedule. Similarly, if the alert is left un-acknowledged for 6 hours, you can have the alert sent to the team lead or manager. This could be an example of how you can ensure that every alert gets attended to

• On-call schedule: This is where you can add all your team members to a rotational on-call schedule. What this means is that the person who is on-call at any given point in time, will receive notifications for any alert that comes in while they are on-call. There isn’t really a best practice for setting on-call schedules as it depends on the criticality of the service and the SLAs promised. But ideally, you wouldn't want one person to be on-call for more than 8-12 hours at a stretch and not for more than a week in one go.

Step 4 - Create an Integration with your monitoring tool

An integration is how your software and/or hardware stack can tell you that something is wrong. Usually, one would set up an observability tool to collect metrics from the tech stack and pass on alerts to Compass if the metrics do something that isn’t expected. For example, if you have a web service that you need up 99.99% of the time, because it is what keeps your websites running, then you will have some observability tool like New Relic or Zabbix watching over the web service all the time. The moment your observability tool detects a blip in the web service, it will send an event to Compass which will create an alert for your on-call team

Compass offers the ability to integrate with more than 150 monitoring and observability tools. Setting up an integration with Compass is very easy. All you have to do is select the tool of your choice, set up the responder team for which this integration is important and follow the steps mentioned in each integration. For most integrations, you can create web hooks using the Atlassian URL end point. This is an example for New Relic.

The next thing for you to do would be to specify what conditions need to be met for an alert to be created or edited or closed. And you can easily set that up using our intuitive interface.

One very important call out is that Compass allows you to set up integrations that map to individual components. This means that any alert that gets generated from the integration always has the affected component displayed along with its deployment history, making it super simple for the on-call responder to investigate the causes of the alert.

Congratulations! And with that, you have set up your integrations!

Step 5 - Configure how you want to receive notifications

The final piece of the puzzle would be for you to specify how you would like to get notified and in what conditions. Compass offers 5 ways in which you can get receive notifications for your alerts and on-call schedules:

1. Voice (phone)

2. SMS (phone)

3. Mobile app

4. Email

5. Slack

The way to set up these preferences would be to go to Settings on the top right and selecting Alert notifications. After you have verified your mobile number, entered your email address(es) and downloaded the mobile app, you are all set to configure your notification preferences.

You can set up notification preferences for 7 different kind of events. In the example below, we are setting up preferences for a new alert. In this scenario, I am choosing to get parallel notifications on email, mobile app and through voice for all incoming alerts. Of course, I receive these notifications only when I am on-call or manually paged by someone else or by a policy.

And there you have it! In 5 simple steps, you have successfully gotten started with Alerts and On-call in Compass! Your tech stack, and your customers will thank you for it 🙂

In the next post

In part 2 of this learning series we will go deeper in to the world of Operations by diving in to some more advanced concepts like:

• Setting up policies

• Setting up automated syncs with Jira Software and Jira Service Management

• Creating Incidents in Jira Service Management, and establishing better collaboration with ITOps teams

• Setting up heartbeats to check if your observability tools are working fine

• Setting maintenance windows during planned downtimes and patching windows

• Setting up a Slack integration for receiving alerts in slack channels

• Creating reports for all your alerting and on-call history