Prometheus Exporter Pro v.3+: add custom metric for monitoring web requests

[#1]

Hello!

In this article we will continue to talk about adding custom metrics in Prometheus Exporter Pro v3+. You can read the basics of adding custom metrics here.

In this article we will discuss how to add your custom metrics to monitor web requests.

You can find the source code for the article here.

You can find the video for this article here.

You can monitor web request out of the box. You can read my article about it.

But suppose it is not enough for you. In this case you can add your own metric to accomplish what you need. In this article I will show how you can do it.

By default we have jira_request_duration_path for Jira, confluence_request_duration_on_path for Confluence and bitbucket_request_duration_on_path for Bitbucket. This metric has default buckets which are set by the prometheus client itself. But suppose you want to set your own buckets. Let's say 4 buckets: 1sec, 5sec, 10 sec and 20sec.

Let's create this metric.

First, you need to create a collector src/main/java/ru/matveev/alexey/prometheus/jira/extension/metrics/requestmetric/RequestMetricCollector.java:

public class RequestMetricCollector extends PrometheusMetricCollector {

    public RequestMetricCollector(RequestHistogram requestHistogram) {
        this.getMetrics().add(requestHistogram);
    }
}

It looks like other collectors. We just register our HistogramMetric.

Then we register our collector in the atlassian-plugin.xml file:

<prometheusRequestMetric key="request-metric-sample-module" name="Sample Request Metric Module"
                             class="ru.matveev.alexey.prometheus.jira.extension.metrics.requestmetric.RequestMetricCollector"/>

As you can see we use here prometheusRequestMetric module, not prometheusMetric module

And then we create our metric:

@Named
public class RequestHistogram extends PrometheusRequestMetric {
    Histogram simpleRequestetric = Histogram.build()
            .buckets(1, 5, 10, 20)
            .name("simple_request_metric")
            .help("Request duration on path")
            .labelNames("path")
            .create();

    public RequestHistogram() {
        super("simple_request_metric", "Request duration on path", true, true);
    }


    @Override
    public Histogram.Timer startTimer(String path, String queryString, ServletRequest servletRequest) {
        return this.simpleRequestetric.labels(path).startTimer();
    }

    @Override
    public PrometheusMetricResult collect() {
        return new PrometheusMetricResult(simpleRequestetric.collect());
    }

}

We extend from the PrometheusRequestMetric class. We define our metric:

Histogram simpleRequestetric = Histogram.build()
            .buckets(1, 5, 10, 20)
            .name("simple_request_metric")
            .help("Request duration on path")
            .labelNames("path")
            .create();

As you can see we define custom buckets with .buckets(1,5,10,20)

All other things look the same as for a usual metric except the startTimer method:

    public Histogram.Timer startTimer(String path, String queryString, ServletRequest servletRequest) {
        return this.simpleRequestetric.labels(path).startTimer();
    }

This method starts the timer for the metric and return this timer. In this method you do all your logic. For example, check if the path is valid or the query string has a certain value for a certain parameter and only in this case you start the timer to measure your metric. Also if you need more than the path and the query string you can use the servlet request to get all parameters for the request.

As the result you will see the following metric:

simple_request_metric_bucket{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",le="1.0",} 2.0
simple_request_metric_bucket{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",le="5.0",} 2.0
simple_request_metric_bucket{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",le="10.0",} 2.0
simple_request_metric_bucket{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",le="20.0",} 2.0
simple_request_metric_bucket{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",le="+Inf",} 2.0
simple_request_metric_count{path="/download/resources/jira.webresources:util-lite/getOptionsFromAttributes.js",} 2.0

That is it. You created your own custom request metric.