Introducing Compass JQL metrics to track Jira issue counts

[#1]

You can track all sorts of things in Jira, so issue counts provide lots of signals about state of your product or team. Given that Compass is the place that customers can implement and track adherence to standards and practices, and since you can attach Compass components to Jira issues across multiple projects, it makes sense to be able to leverage issue counts in Compass scorecards via metrics.

There are lots of use cases:

1. Security Issue Tracking: Security Issue Tracking: By monitoring open security issues, teams can proactively address vulnerabilities and enhance component security

2. Performance Monitoring: Track performance-related issues, such as those labeled with "performance," to identify components that may need optimization.

3. Bug Tracking: Keep a count of open bugs over time to assess the stability of different components and prioritize bug fixes.

4. Feature Request Management: Monitor the number of open feature requests to understand demand and prioritize development efforts.

5. Technical Debt Assessment: Track issues labeled as "technical debt" to manage and reduce technical debt over time.

6. Compliance and Auditing: Count compliance-related issues to ensure that components meet regulatory requirements and are audit-ready.

7. SLA breaches: Track outstanding customer-reported issues to make sure you’re meeting your SLA commitments to them.

8. Release Readiness: Track issues related to upcoming releases to ensure all critical issues are resolved before deployment.

To solve for these use cases, you can now track issue counts in Compass via custom JQL metrics.

Custom JQL metrics in Compass empower teams to tailor their tracking and reporting to their specific needs. By leveraging these metrics, teams can create scorecards that provide valuable insights into their projects, helping them to standardize processes, increase efficiency, and make data-driven decisions. The ability to customize metrics means that teams can focus on what matters most to them, whether it's improving security, enhancing performance, or managing technical debt.

How to get started

The high level flow to get started is:

1. Create a custom JQL metric definition

2. Attach it to components

3. Associate issues with components to populate the metric

4. Use the metric in scorecards

Creating a custom JQL metric definition

1. From the top navigation, select Create → Create metric

2. Give the metric a name, and toggle the “This is a JQL metric” option

3. Input your JQL query

4. Click Create metric

A note about default component scoping

Keep in mind that Compass will prepend an extra condition in order to scope the issue count to that specific component.

For example:

• If the definition query is issuetype = Bug AND statusCategory != Done, and
• you attach that metric to a component named magnetic-api, then
• the query for that component will be (component = "magnetic-api") AND (issuetype = Bug AND statusCategory != Done).

This means that, by default, Jira issues need to be associated with components in order to be counted.

However, if component scoping is undesirable, you can edit the query on a per-component basis. (We also plan to introduce additional scoping mechanisms in the future.)

Attaching and populating the metric for each component

Now that you have created your metric definition, you can to attach it to relevant components and set the credential to be used to call Jira so that values can be populated.

Why do I have to set a credential?

Each request to Jira for issue counts must be made in the context of a user, ensuring that the data retrieved respects the permissions and visibility settings of that user. This approach helps maintain data integrity and security, as it prevents unauthorized access to Jira data.

You can do this one of three ways:

From the component page

1. Go to a component’s page

2. Click the + Connect metric button at the bottom of the page

3. Select your metric from the dropdown

4. Confirm your user as the credential

The metric is now attached and will display an initial value within an hour. It’ll refresh every day.

Remember that Compass automatically scopes the query to that component, so if there are no issues associated with that component at all, an error will display. Associate issues with that component by setting the component field, or, if component scoping is undesirable, you can edit the query on a per-component basis - see below.

Via a scorecard

When you use a custom metric in a scorecard, there is an option to automatically attach it to applied components:

Note: Because JQL metrics require a credential, you still need to set the credential for each component from each component’s page. However, you can delegate task to each team to do for their own components. (We plan to add bulk actions to make this easier in the future.)

Via the GraphQL API

You can use the createMetricSource and updateJqlMetricSourceUser mutations to perform the above steps automatically via API:

• https://developer.atlassian.com/cloud/compass/graphql/#mutations_createMetricSource

• https://developer.atlassian.com/cloud/compass/graphql/#mutations_updateJqlMetricSourceUser

Customizing the query on a per-component basis

As mentioned above, there is an option to let users customize the query for their components, just in case they need to tweak it for their specific situation.

To enable this, check the “Allow component owners to customize a metric’s query for their components” checkbox on the metric definition. (Note: this can’t be changed once saved.)

Once that setting has been enabled, users can edit the query for that component by clicking the three-dot menu on the metric card on the component page and selecting Edit JQL.

Using the metric in scorecards

Now that your JQL metric has been created, attached to components, and data is flowing, you can use it with a scorecard just like any other metric.

See https://support.atlassian.com/compass/docs/use-metrics-with-scorecards/

Give us your feedback

We’ll continue to iterate on this new feature, so we welcome your feedback. Thanks!