An unofficial way to monitor a JSM mail handler for errors

[#1]

Howdy, everyone!

This post is intended to help people monitor their Service Projects for mail handler errors that may not be noticed until a manual check of the Processing log page is performed.

This is also very requested by many of you and after playing with Jira Automation and API for a while now, I finally found a way to make this happen within Jira.

How does this work?

Jira Service Management has an internal endpoint that programmatically returns the same information we see on the Processing log page. The idea here will be to create a scheduled automation rule that will check this endpoint for error logs, format them and forward this to the email address you specify.

But before we proceed, there are a few things that are important to mention:

• It will be necessary to have an Atlassian account with a Jira Service Management license and Project Administrator permission in the Service Projects you want to monitor the mail handlers;

• It is highly recommended for this account to be a service account because its API Key will be exposed in one of the rule’s components.

How can we build this rule?

To ease this guide, we will split it into four parts.

Step 1 - Discovering the endpoint

Each Jira Service Management mail handler has its unique Channel Identifier, which can difficult things a bit, but stick with me as we make this into a piece of cake.

1. Go to your Service Project mail handlers settings:

   1. For company-managed: Project Settings > Email Requests

   2. For team-managed: Service Project settings > Channels > Email

2. Open your Browser’s Developer Tools and switch to the Network Tab

3. Click on View logs for the mail handler you want to get the channel identifier

4. This should record a few endpoint requests, but we need to find one similar to:

   https://instance.atlassian.net/rest/jira-email-processor-plugin/1.0/mail/audit/process/CHANNELXXXXXXXXXXXX?limit=50&page-1

5. When you find it, copy the CHANNELXXXXXXXXXXXX to somewhere you can easily get it later like a text editor

To be sure you found the correct endpoint, you can simply copy the endpoint and open a new tab, the result should be a JSON structure similar to this:

Here is a quick recording showing how to achieve this:

Step 2 - Generating an API Key

Even though we have found the endpoint to retrieve the logs, we still need to be authenticated to access this information and to achieve that we will need an Atlassian account with Project Administrator permission, a Jira Service Management license, and its API Key. After deciding which account to use, follow this guide to create its API Key.

Step 3 - Encoding credentials

You can encode the credentials yourself by opening your Browser's Developer Tools. In the Console tab, type in the following and click enter:

encodedData = "Basic " + window.btoa('ATLASSIAN_ACCOUNT_EMAIL:API_KEY')

You'll need to substitute both the ATLASSIAN_ACCOUNT_EMAIL and API_KEY with the chosen account credentials, like this:

Copy this value (without quotation marks) to somewhere you can easily get it later like a text editor.

Step 4 - Build the automation rule

The final piece in this puzzle… the automation rule. Go to your Project Settings > Automation and start a new rule in the following components:

Scheduled Trigger: The frequency will be up to you, but for the sake of the example, let’s set it to run every hour.

Create variable: This variable will be used in the endpoint we saved earlier as an epoch time. This is necessary to limit the logs by date, otherwise, you would keep retrieving the same errors every day.

• Variable name: epochtimestamp (suggestion)

• Smart value: {{#now}}func=minusHours(1), format="toMillis"{{/}}

  • The minusHours(n) is related to the frequency selected for the trigger, which is why we are using minus one hour. Replace Hours with Days/Weeks and n with the number of the selected unit you want to look back at.

Send Web Request: The component where all things come together. Here we will programmatically simulate the action to open the Processing log page.

• Web request URL: {{BaseURL}}/rest/jira-email-processor-plugin/1.0/mail/audit/process/CHANNELXXXXXXXXXXX?from={{epochtimestamp}}&statuses=FAILED

  • Replace CHANNELXXXXXXXXXXX with Channel Identifier we got at the end of Step 1.

  • ?from={{epochtimestamp}} is being used as a time window to avoid retrieving old errors

  • &statuses=FAILED was added so we only retrieve email logs with the FAILED tag.

• Headers:

  • Key: Authorization

  • Value: Basic c2VydmljZWFjY291bnRhdGxhc3NpYW4uY29tOmNUVnlNbEJxTWtSNWNXaGhiVzFtWkZWeg== (Same credentials saved at the end of Step 3)

• HTTP Method: GET

• Wait for response: Flagged

Advanced Compare Condition: A simple verification to avoid sending notifications without errors.

• First value: {{exists(webResponse.body.data.first)}}

  • The smartvalue function exists() returns true if any content is returned for the webResponse.body.data.first smartvalue

• Condition: equals

• Second value: true

Send Email: The last component of our rule, responsible to notify the administrators about the errors in your Jira Service Management mail handlers.

• To: Here you can select Jira groups or add email addresses manually

• Subject: {{webResponse.body.data.first.mailChannelName}} - Mail logs (suggestion)

  • The smartvalue is getting the mail handler address retrieved in the Send Web Request action and appending it to the subject of our notification

• Content:

  {{#webResponse.body.data}}
  <b>Error Message</b>: {{message}}
  <b>Sender</b>: {{fromAddress}}
  <b>Subject</b>: {{subject}}
  <b>Event Time</b>: {{createdDateTime}}
  ---
  
  {{/}}

  • I will not dive into these smartvalues in this article, but I can tell we are using list expressions and HTML here to format our notification based on the Send Web Request response. If you want to learn more about it, you can check Smart Values - Example with lists.

Finish your rule by giving it a name, and be sure you leave it enabled.

Example

After that, the result should be similar to the image below which matches perfectly with what we saw at the beginning of this page:

Additional Resources

It isn't as good as a native solution, but I hope it will be useful for many of you until Jira Service Management makes it natively available.

Here are some additional links for anyone interested in learning more about Jira Automation, as well as two rules that you can easily import into your Atlassian site based on what we reviewed:

Automation Rules

• Rule #1 - Alert Admins: This is exactly what we saw here, ready to be imported. ( Download Link)

• Rule #2 - Alert Sender: Following the previous rule, this one will notify the senders if their emails were not processed. (Download Link)

Both rules require you to replace the Credentials and Channel Id.

Related Feature Request

• JSDCLOUD-2517

Official Documentation

• Atlassian Account - Create an API Token

• Cloud Automation - Key Concepts

• Cloud Automation - Import and Export Rules

• Cloud Automation - Smart-Values: Conditional Logic

• Cloud Automation - Smart-Values: Date and Time

• Cloud Automation - Smart-Values: Lists

• Cloud Automation - Smart-Values: Examples with Lists

External Documentation

• Finding Your Browser's Developer Console

[Henri Volk _amily_]

JUST FIX THE DESIGN ISSUE FROM https://jira.atlassian.com/browse/JSDCLOUD-2517

[Wagner Filho]

Great article, Yuri!! 

[Milford Mitchell]

Even though we have found the endpoint to retrieve the logs, we still need to be authenticated to access this information and to achieve that we will need an Atlassian account with Project Administrator permission, a Jira Service Management license, and its API Key. After deciding which account to use, follow this guide to create its API Key. ForAgentsOnly

[Brittany Northwood]

While I love that this there is a workaround available to our license, I worry that this article is going to slow down an actual enhancement. Please do not let it.

I am the service desk scrub that has to check the logs every day, I don't have either the permissions necessary to make this happen, or the technical background. I would have to take this article to a dev or something to understand how it is used. 

Don't get me wrong, I am going to bring this to them, but we want our devs working on things our clients are paying for, not trying to force something we are paying for to work for us. 

Every single organization shouldn't have to do this complicated step themselves. Please just build it in so people don't have to dedicate this time. All I want is a dropdown box in the email config section, "Send notification of email failures to [Select User here]" and have that just send an email or notification whenever this happens. 

That said, yay for me! (maybe, if I can get my team to do this for me). 

[Warren Turner]

Hi Yuri,

Really helpful, despite it being a workaround and should be core functionality, this will help my team so they don't have to remember manually interrogate the log on a regular basis.

How can we ensure that the web request results only return items that have been logged/rejected since the last time the rule run?

I don't want to see the same email address appearing every couple of hours if the team have already successfully added an address as a customer so their requests can be accepted successfully thereafter.

[Yuri Moura]

Hi Warren,

Thank you for your feedback! I totally agree this feature should be core functionality, especially because this workaround still has limitations and it is not something easy to configure.

Regarding your concern, there are two steps where your inputs must match to avoid receiving the same logs between executions.

The first one is in the trigger, where you define the frequency the rule should run. In this example, I defined that my rule should trigger every hour.

The second one is in the epochtimestamp variable creation. The input here must match the trigger frequency because the variable is responsible for defining how much time we want to look back in the logs from now.

So, in short, if we are executing a rule every hour, my variable must look back an hour from now in the logs to not overlap the last execution, right? For this, we use the following expression:

{{#now}}func=minusHours(1), format="toMillis"{{/}}

My recommendation is to try putting the rule into action with the trigger occurring once every hour and leaving the variable as it is. If this works, you can later experiment with the trigger and the variable to find a frequency that is more appropriate according to your requirements.

I hope this helps, Warren! 

[Warren Turner]

@Yuri Moura - thanks for the feedback.

I have adjusted it back to the baseline 1hr on the trigger and the epochtime var yet I am still seeing it repeat the same address despite the log of the failure being more that 1hr previously to now()-1hr - as you can see this shows event times from Thursday last week yet the rule was manually run at 9.21am today:

Any further advice would be appreciated.

[Emmy So]

Hi Yuri,

Thank you for the very helpful article.

But I keep getting this error when I try to setup the automation job.

Can you please help?

Thanks so much!

Emmy

[Rajaa Jabir]

Hi Emmy,

1. Use a service account with Jira service management license and add the account as project admin.

2. Login with service account to Jira, and generate the API token.

3. Then use the service account email and generated API in step 3- encoding credentials, and then follow the rest of the steps for the automation rule creation.

Regards

[Carol K_ Guss]

This cutoff concentrates nicely for us, favor your heart! Going toward an essentially indistinguishable issue here. Help is respected. Flying Together

[Khai Shaun Ng]

Hi, is it possible to extend this logic to include the Raw email message as an attachment?

[Nena Kruljac]

Hi, 

thank you for this workaround solution :-). It would save us time and these checks could be done more often then now.

We will try it (hoping soon). But what about security? How secure is this - can someone intercept traffic and misuse the information somehow?

[Kim-Stacey Kidder]

@Yuri Moura Hi! Thank you for this! 

For status tags that contain more than 1 word, what would the web request url look like (the tag portion that is)?: 
 
1-word example: 

&statuses=FAILED

---

2-words (example: CONDITION PENDING): 

&statuses= ???

Would you use CONDITION_PENDING ?

---

Thanks in advance! 

[Kim-Stacey Kidder]

Update: CONDITION+PENDING worked for me 

[pmtorres]

Great guide, we really needed this!

In case that you want to monitor more than one status (e.g. CONDITION PENDING and FAILED), the web request URL parameters are:

CHANNELXXXXXXXXXXX?from={{epochtimestamp}}&statuses=FAILED&statuses=CONDITION+PENDING

 It is not obvious as the parameter statuses is in plural form.