It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Monitoring of Atlassian applications with Prometheus

Hi all,

In this article we will talk about real-time monitoring of Atlassian applications.

But first, let us define what monitoring is and why it is important to monitor Atlassian applications.

Monitoring is used for measuring performance and availability of software applications. Another words, it visualizes the current state of an application in the real-time mode.

When is it useful to monitor Atlassian applications?

Here are a couple of examples:

  • You want to know how an Atlassian application update or a plugin update influences performance.
  • You want to know hardware or memory parameters during certain conditions or certain time intervals. For example, how the number of users or day time influences performance.
  • You want to monitor how active your users are. For example, how many issues are created within a day or an hour.
  • You want to be notified if the license for an Atlassian application or a plugin is about to expire.
  • You want to know how much disk space is taken by attachments.

 It is also important not only to be able to view information about the current state of an application, but also receive notifications if something went wrong.

Solution architecture

Prometheus can monitor thousands of services. Prometheus collects metrics as frequent as you defined, usting the pull-model, and stores collected metrics in the database. Also there is the Alert manager, which can send notifications to different channels, if an event has been triggered.

Prometheus exporter is an agent, which collects metrics from entities (a whole server or an application). Prometheus has rich capabilities for extending, that is why there are exporters for almost all popular applications.

[ jira | confluence | bitbucket | bamboo ]_exporter works the following way:

  • collects information from the database of an Atlassian application;
  • collects information from Java API;
  • intercepts HTTP requests and collects information based on the requests;
  • collects JMX metrics.

You can find an example of Prometheus/Grafana installation in Docker here. This example was kindly provided by Stefan Prodan.

Exporter installation and setup

First, you need to install the plugin for your Atlassian Application. You can find all plugins here. When the plugin has been installed, you can receive metrics by baseUrl/plugins/servlet/prometheus/metrics url.

You also can set up security keys.

Then you should add the reference to the metric collection servlet to the prometheus.yml. For example, for Jira it would look like this:

- job_name: 'jira'
scheme: https # change to http if don't you have https
metrics_path: '/jira/plugins/servlet/prometheus/metrics'
static_configs:
- targets: ['myhost:2990'] # Jira host and port you serve

 Installation and setup is finished. You can collect metrics.

Metrics

You can find the whole list of metrics in the documentation for the plugins: JiraConfluenceBitbucketBamboo.

Query examples:

  • Uptime:

    jvm_uptime_gauge{job="jira"}
  • Server response time:

    (sum(rate(jira_request_duration_on_path_sum[1m])) by (path)) / (sum(rate(jira_request_duration_on_path_count[1m])) by (path))
  • Total number of issues:

    jira_total_issues_gauge
  • Dashboard view number:

    sum(increase(jira_dashboard_view_count[1h]))


More information about queries you can find here.

Grafana dashboards

You can use Grafana to visualize metrics stored in Prometheus. There are a couple of example dashboards in the official site.

You can find a dashboard for Jira here

e4utslhkyv8wb8hhz0rftlsil0c.png

Here is a dashboard for Confluence:

2018-06-07 07.00.46.jpg

Features

There are plans to let users define their own metircs, using the Groovy language. You will be able to collect your custom metrics not only from the Atlassian applications but also from other plugins. Also there will be metrics collected from Jira Service Desk.

All Prometheus exporters for Atlassian applications are open source and continuously gaining new features.

If you have ideas what features should be included in the plugins, kindly create tickets (JiraConfluenceBitbucketBamboo) or open pull requests.

Conclusion

If your Atlassian instances are large and you need to ensure their stability, then you need to use monitoring. It is up to you to decide how to do it. Prometheus is one of the possible solutions to accomplish this task.

Thank you!

17 comments

Great post! thanks a lot for sharing this info. I have implemented it in our Jira instance and it works very well. One question, do you know if it's possible to show in the Requests duration gadget the info from all of our apps? e.g. eazyBI, Structure, etc.

Regards

Thank you for the comment! The guy who develops the plugin said that he will add the ability to get metrics from other plugins. 

Great!!!! thanks.

@Carlos Aparicio [DEISER] Hi Carlos. Currently I expose only 1st level of paths: rest, servlet, projects, etc. But the next release will also custom paths with n levels. It will fully cover your needs.

Thanks @Andrey Markelov That will be really useful!

regards 

Interesting.  We are building a similar solution based on ELK stack based on presentation we saw at Summit last fall.  Even though we aren't done with the visualization part, just being able to have all out logs routed and queryable in Elastic search has already help us track down a couple issues.

@Daniel Holmes Daniel, sure it will work but it doesnt allow combine with system metrics like swapping etc. Also prometheus has alert manager. Graylog has it but also not fully specialised for that.

Elastic has a layer called Beats that helps to collect OS level metrics as well as JVM and DB specific data.  There is an alerts module called X-Packs that we haven't yet started to look at yet.  Internally we were also building a common Elastic infrastructure so this path seemed to be reasonable for us to pursue for the moment.

But definitely agree that regardless of the tool, you need to have this type of correlating data visualization to understand the system performance.

@Alexey Matveev [cPrime] I'm having a tough time creating a datasource. Could you help me ?

I am doing everything on localhost.

I've selected Prometheus as a type. what should be my url ? It says default prometheus server url on 9090. But I have installed only plugin - not server.

Great article, thank you!

Fantastic Article! Cheers!

I agree it is a wonderful article.

Any expected time line when we can expect the feature of customizing the metrics as quoted in the article in Features section as-

"There are plans to let users define their own metircs, using the Groovy language. You will be able to collect your custom metrics not only from the Atlassian applications but also from other plugins. Also there will be metrics collected from Jira Service Desk."?

Very nice, do you guys have any recommended Grafana dashboards asides from the stock ones at their site?

Hi @Andrey Markelov

thanks for this great plugins. I'll read the metrics with the existing telegraf plugin. But the metrics URL is secured and needs a user authentication in JIRA. In Confluence and Bitbucket, it's working great.

Do you know a way, to disable security for the metrics URL in JIRA?

Thanks in advance.

Regards

   Steffen...

Can we use this on data center?

Yes, it can be used on Data Center, though there is not label for it.

hmedia01 I'm New Here Jun 16, 2019

No metrics for Jira Projects (Update Issues per Project)

Server edition (latest)

Comment

Log in or Sign up to comment
Community showcase
Posted in Jira

Calling all Jira Cloud users! Give us feedback on our exploration of a new navigation.

Hi everyone! My name’s Matt and I’m a product manager at Atlassian. I work in the navigation & findability space for all our Jira Cloud products. We’ve been working on trying to improve the exp...

883 views 14 12
Join discussion

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you