Jira messages from AWS SES marked as Spam in Google

[Bob Kleinberg]

Problem:

We have a Jira Data Center instance hosted on an AWS EC2 VM which uses AWS Simple Email Service (SES) for outgoing mail (Outgoing mail SMTP Mail Server).  Here is the test message output:

This is a test message from JIRA. 
Server: AWS SES
SMTP Port: 465
Description: 
From: jira@jerseystem.org
Host User Name: <removed>

Our Jira users have been receiving messages in the Google Email accounts sent from Jira (e.g., assignment notifications, directed messaging in comments, watching, etc.) marked by Google as Spam. 

This began early this early.   We have had to tell to them to go into their web email browser and mark messages as "Not Spam".  Of course this approach is fraught with problems.   We are constantly getting new users in our system and the message is not passed on (in reality, nor should we expect it to).   I've posted an Help Request to Atlassian (https://support.atlassian.com/requests/GHS-273601/) and they have been a help in general in identifying the general cause of the problem.   However, I wanted to document for the community the specific steps I took to address the problem.

The basic problem traces back to the change Google made in February 2024 regarding enforcement of Domain-based Message Authentication, Reporting, and Conformance (DMARC), a description of which can be found at https://dmarc.org/ .  One of the indications that a DMARC problem exists is in the appearance in Google emails of the phrase "via amazonses.com" next to the sender of the email (in our case, jira@jerseystem.org). 

If you are having emails marked as Spam by Google and/or seeing that "via" phrase next to the sender (see picture below), then the following approach can help you.

Approach:

To address this issue, you or your team must have Admin access to AWS SES and your organization's DNS server.  It also helps if you / your team have Jira admin access to send test messages and Google Console (specifically, Email Log Search) to check their status.

In summary, since we are using jira@jerseystem.org as the named "From user" (user "jira" in our domain jerseystem.org) but not originating from that email user in jerseystem.org, but rather using AWS SES as an intermediary sender to send on behalf of that user,  in order to pass the DMARC test, we had to align the FROM DOMAIN in our AWS SES jira@jerseystem.org identity to a real sub-domain in our jerseystem.org DNS.  This alignment of the "From" in the message to the actual sending domain (the "FROM DOMAIN" field) allows the DMARC test to pass.  The following instructions describe how this is done.

In AWS SES:

1. Login into your AWS admin account
2. Find the Amazon SES service
3. Navigate to Identities (https://<your AWS region>.console.aws.amazon.com/ses/home?region=<your AWS region>/identities
4. Find the identify which you are using to send your email message from Jira (ours is jira@jerseystem.org) and click on it.
5. Scroll down to the section on "Custom MAIL FROM domain.  If you have not configured the MAIL FROM domain. then this section will show "Not started" under the MAIL FROM configuration column and the other two columns - MAIL FROM domain and Behavior on MX failure - will be blank.
6. Click the Edit button on the right.
7. In the General details display that pops up, click on the box next to "Use a custom MAIL FROM domain".    A MAIL FROM domain sub-window appears.
8. In the box "Enter a subdomain", enter the subdomain in your email domain that exists (or you will create) and that will get two additional DNS records.  For us, since our jira instance has a DNS address of jira.jerseystem.org, the answer was pretty easy - we used "jira" in that box. 
9. Unless your DNS admin says otherwise, I'd leave the default check option for Behavior on MX failure as "Use default MAIL FROM domain".
10. Then same changes.
11. Coming back to the Custom MAIL FROM domain, click on Publish DNS records.  This will open up a window with the two records to add to the DNS record for your identity you are using (for us, jira@jerseystem.org).
12. Click on the "Download .csv record set"
13. Pass that information to the DNS administrator
14. You should also check the DMARC policy, "_dmarc.<your domain>" (ours is _dmarc.jerseystem.org.  Ours is set to "v=DMARC1; p=none;".  Download and send that to your DNS administrator. 

In DNS:

1. Log into your DNS service
2. Find / create the subdomain identified in step 8, above.  
3. Add to that subdomain record the the two records passed in the csv record set.
4. Check the DMARC policy for the domain and set accordingly.
5. Publish

Here's what our sending Identity page looks like for the affected areas:

At this point, in may take a while for the changes to propagate through the DNS environment.  But, maybe a day later you can check on status.   Here's what you can do to check:

In AWS SES:

Find the identity you used above.  Once the information is properly propagated, you will see what we have above, specifically:

• MAIL FROM configuration column: Successful (with a little checkmark)
• MAIL FROM domain: your entered subdomain in item 8 in the AWS SES configuration above (recall, mine was jira.jerseystem.org)
• Behavior on MX Failure: whatever you selected in item 9 in the AWS SES configuration discussion above.

In Jira and your GMail web browser

Use the System admin function Mail | Outgoing mail to send a Test message using the associated SMTP Mail server to yourself.  When it appears in your Google Email, check to see if the phrase "via amazonses.com" no longer appears next to the From email address, as in picture below:

In Google:

This will get a little trickier since finding "Spam" messages are tough.   You may need to wait a few days for this test. 

After about 5 days, Filter on sender = <your jira email sender, e.g., mine is jira@jerseystem.org> and use a custom date range of at least 14 days that includes a few days on either side of the date you made the DMARC change. 

Then download the results to a csv file. In that file, look for receiving users who have "Marked Spam" as the event status prior to the time when you made the change and then the same user after, but as close as to as possible to, the date the change became effective.  You should see "Delivered" as event status instead.   

For this test, it is possible that the user could have received the Marked Spam email, marked as Not Spam and then subsequent emails would be okay.  So it's best if you check the complete history for the email with event "Marked Spam" to make sure that the user did not subsequently change the status.  If they did not, then the subsequent "Delivered" is evidence that the DMARC change was effective.