Introducing: Derived Metrics!

[#1]

Compass provides the most value to your organization when it's populated with components, events, and metrics. But creating both events and metrics manually can be a burden. We've heard your feedback, and are happy to officially announce derived metrics: metric sources that are automatically calculated based on your event data!

Previously, to insert a metric value, you would have to provide it directly via our `insertMetricValue` API. We noticed that for many our built-in integrations, we were duplicating logic to calculate and send similar metric values. Now, you can simply create a derived metric source, and its value will automatically be calculated by Compass, based on the relevant events on that component.

For example, when Compass receives a deployment event, it automatically adds new values for the deployment frequency and deployment time metrics.

Here's the full list of built-in metrics that are automatically derived from events:

Event type	Metrics derived from event
Deployment	Deployment frequency Deployment time
Build	Build success rate Build time
Incident	Mean time to resolution (MTTR)

Setting up a derived metric source

• When you connect a metric from the above list to a component via the Compass UI, it will automatically be created as derived
• When creating a new metric source from the API, pass the `derived: true` option to your `createMetricSource` input

For example:

{
 "input": {
  "componentId": "<Your componentARI here>",
  "metricDefinitionId": "ari:cloud:compass::metric-definition/builtin/weekly-deployment-frequency-28d",
  "externalMetricSourceId": "test123",
  "url": "https://google.com",
  "derived": true
 }
}

Updating a derived metric

You cannot insert metric values into a derived metric source manually. They are only recalculated based on events.

After creating a derived metric source on a brand new component, it'll look like this:

It communicates that the value will be calculated by Build events, so for the metric to have value, you need to add Build events to your component via the API. Calculations differ by metric, but in general, we tend to look at events in completed states. For Build Success Rate, we don't take into accounts builds that are currently running, only builds that have completed and have succeeded or failed. For Deployment Frequency, we only look at successful deployments. Please look forward to more in-depth documentation on how each derived metric is calculated.

Thankfully, many of our integrations such as GitHub and Bitbucket now benefit from derived metrics, and will create metric sources that are automatically calculated based on the events they provide. This feature makes it easier for app developers to quickly utilize key built in metrics without having to write calculation code themselves.

Future plans

We welcome your feedback! Let us know where derived metrics help your workflows, and where they could be improved. While our list of derived metrics is small right now, we have plans to add many more, such as Incident Frequency, Change Failure Rate, and Pull Request Cycle Time. Our vision is for Compass to be driven by rich event data, so we definitely want to know which events and metrics would best help you to monitor and assess your organization's infrastructure.

Cheers,
Henry

[Henry Olson]

Here's a video demonstrating how to set up derived metrics on your components:
https://www.loom.com/share/9890ae66e90d4cb58b1669ae9a6a67cd

[Jan-Hendrik Spieth]

Thanks for the helpful video, @Henry Olson! I'm having trouble finding the documentation for e.g. deployment events, or also build events. Could you point me to the right spot, plz?

[Daniel Konold]

any news when Change Failure Rate will be a derived metric? Or any hints how to get the correct data from bitbucket into compass?