Bulk Notifications are send with a delay of up to one hour

BJW

Environment:

• Jira On Prem 8.4.2
• running on Windows Server 2012 R2
• Bulk notifications are enabled

 

Problem:

• Bulk notifications should be send out every 10 minutes, but actually are sent out one hour later (not correct)
• notifications for @-mentions are directly sent out (correct)

 

Things tried so far:

• Mail queue in Admin-Interface is not showing the bulk-mails as queued. In case we have a queued @-mention notification it is shown correctly in the queue up until sending
• atlassian-jira-outgoing-mail.log shows no relevant information
• turned on Debugging for outgoing mails -> log shows no relevant information
• from Admin-Interface, changed Mail Queue Service from every minute (via cronjob) to every 15 minutes (via text selection) -> behaviour is correct. @-mentions are queued up for 15 minutes and then send out. No affect on bulk notifications.
• idea: problem not related to mail queue, but rather to creation of mail inside Jira
• verified atlassian-jira.log -> nothing special
• analyzed usage of VM with VisualVM -> Heap size is 10GB (Min/Max), visualVM shows that only 4GB are in use. This also confirms that we do not see any OutOfMemory Exceptions.

 

 

I found this article describing issues with SMTP connection due to IPv6 issues.
https://confluence.atlassian.com/jirakb/unable-to-connect-to-mail-server-due-to-ipv6-issues-on-jvm-637731067.html

-> I have not applied the fix on our side, as... 1) we do not see those error messages in the log and 2) sending of direct mails is working correctly

 

I also checked the release notes of upcoming Jira Releases, but I was not able to find any issue directly or potentially indirectly linked with this kind of issue.

 

Do you have an idea why we are experiencing this issue?

Do you have any idea how we can proceed analysis to trace down the root cause?

 

Thanks!

BJW

Hi JP, 

thanks for your feedback. I have already verified the configuration there and changed from cron to options-list. But unfortunately no success. (see my initial post).

.....but - I was able to find the root cause!

While checking this article https://confluence.atlassian.com/confkb/confluence-notification-mail-slow-to-arrive-or-having-huge-delay-833926304.html I stumbled upon the debug parameters com.atlassian.confluence.service.services.mail and com.atlassian.mail.queue.
Having those two active for some time, I saw that we have one mailbox configured for JETI / Mail to issue causing masses of errors. The primary mail address of this inbox changed and was not reflected in JETI configuration.
2020-03-08 08:23:05,505 INFO [abc@xyz.com] Caesium-1-3 abcdefg abc@xyz.com.com abc@xyz.com.com[11201]: Cannot handle message as the recipient(s) (123@xyz.com) do not match the catch email abc@xyz.com

After correcting this, the performance of the mail notifications was again normal and as expected (@-mention within one minute, bulk notification within 10 to 15 minutes.

Cheers, B