An unofficial way to monitor a JSM mail handler for errors

Howdy, everyone!

This post is intended to help people monitor their Service Projects for mail handler errors that may not be noticed until a manual check of the Processing log page is performed.

image-20220909-061357.png

This is also very requested by many of you and after playing with Jira Automation and API for a while now, I finally found a way to make this happen within Jira.

How does this work?

Jira Service Management has an internal endpoint that programmatically returns the same information we see on the Processing log page. The idea here will be to create a scheduled automation rule that will check this endpoint for error logs, format them and forward this to the email address you specify.

But before we proceed, there are a few things that are important to mention:

  • It will be necessary to have an Atlassian account with a Jira Service Management license and Project Administrator permission in the Service Projects you want to monitor the mail handlers;

  • It is highly recommended for this account to be a service account because its API Key will be exposed in one of the rule’s components.

How can we build this rule?

To ease this guide, we will split it into four parts.

Step 1 - Discovering the endpoint

Each Jira Service Management mail handler has its unique Channel Identifier, which can difficult things a bit, but stick with me as we make this into a piece of cake.

  1. Go to your Service Project mail handlers settings:

    1. For company-managed: Project Settings > Email Requests

    2. For team-managed: Service Project settings > Channels > Email

  2. Open your Browser’s Developer Tools and switch to the Network Tab

  3. Click on View logs for the mail handler you want to get the channel identifier

  4. This should record a few endpoint requests, but we need to find one similar to:

    https://instance.atlassian.net/rest/jira-email-processor-plugin/1.0/mail/audit/process/CHANNELXXXXXXXXXXXX?limit=50&page-1
  5. When you find it, copy the CHANNELXXXXXXXXXXXX to somewhere you can easily get it later like a text editor

To be sure you found the correct endpoint, you can simply copy the endpoint and open a new tab, the result should be a JSON structure similar to this:

image-20220909-061027.png

Here is a quick recording showing how to achieve this:

Step 2 - Generating an API Key

Even though we have found the endpoint to retrieve the logs, we still need to be authenticated to access this information and to achieve that we will need an Atlassian account with Project Administrator permission, a Jira Service Management license, and its API Key. After deciding which account to use, follow this guide to create its API Key.

Step 3 - Encoding credentials

You can encode the credentials yourself by opening your Browser's Developer Tools. In the Console tab, type in the following and click enter:

encodedData = "Basic " + window.btoa('ATLASSIAN_ACCOUNT_EMAIL:API_KEY')

You'll need to substitute both the ATLASSIAN_ACCOUNT_EMAIL and API_KEY with the chosen account credentials, like this:

image-20220909-060900.png

Copy this value (without quotation marks) to somewhere you can easily get it later like a text editor.

Step 4 - Build the automation rule

The final piece in this puzzle… the automation rule. Go to your Project Settings > Automation and start a new rule in the following components:

Scheduled Trigger: The frequency will be up to you, but for the sake of the example, let’s set it to run every hour.

image-20220911-221231.png

Create variable: This variable will be used in the endpoint we saved earlier as an epoch time. This is necessary to limit the logs by date, otherwise, you would keep retrieving the same errors every day.

  • Variable name: epochtimestamp (suggestion)

  • Smart value: {{#now}}func=minusHours(1), format="toMillis"{{/}}

    • The minusHours(n) is related to the frequency selected for the trigger, which is why we are using minus one hour. Replace Hours with Days/Weeks and n with the number of the selected unit you want to look back at.

image-20220911-221239.png

Send Web Request: The component where all things come together. Here we will programmatically simulate the action to open the Processing log page.

  • Web request URL: {{BaseURL}}/rest/jira-email-processor-plugin/1.0/mail/audit/process/CHANNELXXXXXXXXXXX?from={{epochtimestamp}}&statuses=FAILED

    • Replace CHANNELXXXXXXXXXXX with Channel Identifier we got at the end of Step 1.

    • ?from={{epochtimestamp}} is being used as a time window to avoid retrieving old errors

    • &statuses=FAILED was added so we only retrieve email logs with the FAILED tag.

  • Headers:

    • Key: Authorization

    • Value: Basic c2VydmljZWFjY291bnRhdGxhc3NpYW4uY29tOmNUVnlNbEJxTWtSNWNXaGhiVzFtWkZWeg== (Same credentials saved at the end of Step 3)

  • HTTP Method: GET

  • Wait for response: Flagged

image-20220911-221425.png

Advanced Compare Condition: A simple verification to avoid sending notifications without errors.

  • First value: {{exists(webResponse.body.data.first)}}

    • The smartvalue function exists() returns true if any content is returned for the webResponse.body.data.first smartvalue

  • Condition: equals

  • Second value: true

image-20220911-221217.png

Send Email: The last component of our rule, responsible to notify the administrators about the errors in your Jira Service Management mail handlers.

  • To: Here you can select Jira groups or add email addresses manually

  • Subject: {{webResponse.body.data.first.mailChannelName}} - Mail logs (suggestion)

    • The smartvalue is getting the mail handler address retrieved in the Send Web Request action and appending it to the subject of our notification

  • Content:

    {{#webResponse.body.data}}
    <b>Error Message</b>: {{message}}
    <b>Sender</b>: {{fromAddress}}
    <b>Subject</b>: {{subject}}
    <b>Event Time</b>: {{createdDateTime}}
    ---
    
    {{/}}
    • I will not dive into these smartvalues in this article, but I can tell we are using list expressions and HTML here to format our notification based on the Send Web Request response. If you want to learn more about it, you can check Smart Values - Example with lists.

image-20220911-221017.png

Finish your rule by giving it a name, and be sure you leave it enabled.

Example

After that, the result should be similar to the image below which matches perfectly with what we saw at the beginning of this page:

image-20220909-070403.png

Additional Resources

It isn't as good as a native solution, but I hope it will be useful for many of you until Jira Service Management makes it natively available.

Here are some additional links for anyone interested in learning more about Jira Automation, as well as two rules that you can easily import into your Atlassian site based on what we reviewed:

Automation Rules

  • Rule #1 - Alert Admins: This is exactly what we saw here, ready to be imported. ( Download Link)

  • Rule #2 - Alert Sender: Following the previous rule, this one will notify the senders if their emails were not processed. (Download Link)

Info icon Both rules require you to replace the Credentials and Channel Id.

Related Feature Request

Official Documentation

External Documentation

 

20 comments

Henri Volk _amily_ September 12, 2022

JUST FIX THE DESIGN ISSUE FROM https://jira.atlassian.com/browse/JSDCLOUD-2517

Like # people like this
Wagner Filho
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 12, 2022

Great article, Yuri!! 

Like # people like this
Milford Mitchell September 26, 2022

Even though we have found the endpoint to retrieve the logs, we still need to be authenticated to access this information and to achieve that we will need an Atlassian account with Project Administrator permission, a Jira Service Management license, and its API Key. After deciding which account to use, follow this guide to create its API Key. ForAgentsOnly

Brittany Northwood October 3, 2022

While I love that this there is a workaround available to our license, I worry that this article is going to slow down an actual enhancement. Please do not let it.

I am the service desk scrub that has to check the logs every day, I don't have either the permissions necessary to make this happen, or the technical background. I would have to take this article to a dev or something to understand how it is used. 

Don't get me wrong, I am going to bring this to them, but we want our devs working on things our clients are paying for, not trying to force something we are paying for to work for us. 

Every single organization shouldn't have to do this complicated step themselves. Please just build it in so people don't have to dedicate this time. All I want is a dropdown box in the email config section, "Send notification of email failures to [Select User here]" and have that just send an email or notification whenever this happens. 

That said, yay for me! (maybe, if I can get my team to do this for me). 

Like # people like this
Warren Turner October 20, 2022

Hi Yuri,

Really helpful, despite it being a workaround and should be core functionality, this will help my team so they don't have to remember manually interrogate the log on a regular basis.

How can we ensure that the web request results only return items that have been logged/rejected since the last time the rule run?

I don't want to see the same email address appearing every couple of hours if the team have already successfully added an address as a customer so their requests can be accepted successfully thereafter.

Yuri Moura
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 21, 2022

Hi Warren,

Thank you for your feedback! I totally agree this feature should be core functionality, especially because this workaround still has limitations and it is not something easy to configure.

Regarding your concern, there are two steps where your inputs must match to avoid receiving the same logs between executions.

The first one is in the trigger, where you define the frequency the rule should run. In this example, I defined that my rule should trigger every hour.

The second one is in the epochtimestamp variable creation. The input here must match the trigger frequency because the variable is responsible for defining how much time we want to look back in the logs from now.

So, in short, if we are executing a rule every hour, my variable must look back an hour from now in the logs to not overlap the last execution, right? For this, we use the following expression:

{{#now}}func=minusHours(1), format="toMillis"{{/}}

My recommendation is to try putting the rule into action with the trigger occurring once every hour and leaving the variable as it is. If this works, you can later experiment with the trigger and the variable to find a frequency that is more appropriate according to your requirements.

I hope this helps, Warren! 

Warren Turner October 24, 2022

@Yuri Moura - thanks for the feedback.

I have adjusted it back to the baseline 1hr on the trigger and the epochtime var yet I am still seeing it repeat the same address despite the log of the failure being more that 1hr previously to now()-1hr - as you can see this shows event times from Thursday last week yet the rule was manually run at 9.21am today:

submission log repetition greater than now -1hr.PNG

Any further advice would be appreciated.

Emmy So November 3, 2022

Hi Yuri,

Thank you for the very helpful article.

But I keep getting this error when I try to setup the automation job.

automation.JPG

Can you please help?

Thanks so much!

Emmy

Like Raul Herrera likes this
Rajaa Jabir November 15, 2022

Hi Emmy,

1. Use a service account with Jira service management license and add the account as project admin.

2. Login with service account to Jira, and generate the API token.

3. Then use the service account email and generated API in step 3- encoding credentials, and then follow the rest of the steps for the automation rule creation.

Regards

Carol K_ Guss December 8, 2022

This cutoff concentrates nicely for us, favor your heart! Going toward an essentially indistinguishable issue here. Help is respected. Flying Together

Khai Shaun Ng December 19, 2022

Hi, is it possible to extend this logic to include the Raw email message as an attachment?

Like Nena Kruljac likes this
Nena Kruljac January 2, 2023

Hi, 

thank you for this workaround solution :-). It would save us time and these checks could be done more often then now.

We will try it (hoping soon). But what about security? How secure is this - can someone intercept traffic and misuse the information somehow?

Kim-Stacey Kidder March 7, 2023

@Yuri Moura Hi! Thank you for this! 

For status tags that contain more than 1 word, what would the web request url look like (the tag portion that is)?: 
 
1-word example: 

&statuses=FAILED


2-words (example: CONDITION PENDING): 

&statuses= ???

Would you use CONDITION_PENDING ?


Thanks in advance! 

Kim-Stacey Kidder March 9, 2023

Update: CONDITION+PENDING worked for me 

Like Troy Anderson likes this
pmtorres May 12, 2023

Great guide, we really needed this!

In case that you want to monitor more than one status (e.g. CONDITION PENDING and FAILED), the web request URL parameters are:

CHANNELXXXXXXXXXXX?from={{epochtimestamp}}&statuses=FAILED&statuses=CONDITION+PENDING

 It is not obvious as the parameter statuses is in plural form.

Like # people like this
Troy Anderson June 20, 2023

One thing I am doing a bit differently is looking at the Connectivity log and not the Process log as our errors are showing up on the connection level.  But I have accounted for that by adjusting the web request URL in step 4 accordingly.  (i.e. "/connectivity/CHANNELXXXXXX").  Still determining if "statuses" is the correct variable to pass, though.  

The Process log has filtering available but the Connectivity log does not, so I believe I am dead in the water - there is no way to pass a variable to the Connectivity log to return only the errors.  If anyone knows otherwise, I would love to hear from you!

Like Kalin U likes this
Emmanuel F-D January 11, 2024

Hi @Yuri Moura 
is it possible to use another method to upload automation rules?
You need to log in with a Google account and request access.
Thanks

Ben Gallimore January 24, 2024

Hi @Yuri Moura 

Is this workaround going to send an email for every entry in the log that has occured in the last hour?

Also as I don't want to clog up the email accounts of my service desk team, I have simply added my automation account as a customer in the project and sent the email to my JSM queue email account, therefore creating a ticket for the team to act on.

Thanks for the awesome workaround!

Bianca Lasalandra February 20, 2024

Hi @Ben Gallimore  - could you please expand on how you updated the automation rule to generate a JSD ticket as opposed to an email? Thanks!

Bianca Lasalandra March 1, 2024

Hello @Yuri Moura

I was hoping you could provide further guidance on how the email log template could be adjusted. 

How could I go about printing the status into the email? What would the variable name be? I tried {{statuses}} and {{status}} but that was unsuccessful. 

Thanks,
Bianca 

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events