JEMH: replace from-address & project routing for very huge amount of projects

Tobias Reibling
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 12, 2014

I have some doubts about the setup of the following Helpdesk scenario with JEMH:

  • approx. 1000 (!) different projects,
  • non-jira email users should be able to create issues, comment and receive notifications on create, resolve, comment
  • routing of issues created into a project should be done via the recipient email address (individual mail addresses, not related to existing JIRA user, not matching the project key)
  • when sending notifications to Non-JIRA-users, the from-address needs to be the respective recipient address (users need to get the email back from the same address they had sent it to)

I have an idea on the general setup for JEMH as regards custom fields, profiles, event listener etc. As we need to route issue creation based on recipient email address, there seems no way to avoid having 1000 project mappings in the profile, and we probably need project mappings in the Profile anyway as we have to control different languages for notifications via the "proxy user". One event listener with one single project mapping should be okay, as there are no differences (using a regex to automatically cover all project keys).

Question1

What is the correct way to handle the notification to non-JIRA-users after issue creation? Profile/Notifications/Notify Users on Issue creation (as says the JEMH "Event Listener" interface itself) or working with a custom event and all the respective setup in the event listener (as say many JEMH tutorials)? And would the answer to this question change after having considered question #2?

Question 2

What would be the correct way to replace the default JIRA outgoing mail from-address by the original (incoming) recipient email address for each project? We can't do it via Profiles where it also seems to be possible (having 1000 profiles that are all the same except for the "from-address" would be crazy). As far as I understand it, this would only be possible using a Project Mapping for each project in the event listener and setting the from address in this mapping - resulting in another 1000 project mappings, but this time for the event listener! Is there any way to avoid that?

Question 3

Assuming that #2 is like it seems to be, would the conclusion be that project routing for issues created based on recipient email is a very bad idea in this case and really needs to be replaced by routing based on project key in email subject (which eliminates the from-address problem and could make project mappings obsolete, too)?

Thanks for any kind of suggestions.

Tobias

2 answers

1 accepted

0 votes
Answer accepted
Andy Brook
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 13, 2014
  • *Project auto assign with a dedicated JIRA domain will work well for any number of projects.* => I played around with the auto project mapping based on the project key in the addressee part of the recipient email (key123@example.com creates issues in project with key123, ...), which would avoid project mappings. (1) But this does not resolve the problem with the notification language (we might convince the customer to drop this) and also does not help with the from-address problem. As long as external users will have to understand that they have to send mails to key123@example.com or key456@example.com or key 678@example.com, (2) of course it is essential that they get any kind of notifications back from the same address.

(1) Right.  Depending how complex you want to get, its conceivable to have a per-project locale setting in the Event Listener, that works fine for one language but not others.

Better than this, something I've been thinking about, is to 'tag' issues that were created by email, with details about where it came from, eg the email TO: and FROM: address.  Then, dynamic locale association could be made in the Event Listener, effectively this would be a Locale to inbound / sender mapping.  Such a feature would allow one Event Listener with a regexp 'all' project match, to support multi-locale template rendering dynamically driven from the inbound mail.  This approach is similar to what has already been said, about the open feature of storing inbound email address 'in' the issue as a custom field value, and driving the from address from that.

(2) Notifications don't need to be sent 'from' the inbound address, its a misconception, they can be sent from some central mail gateway for simplicity reasons, with a reply-to header.  Currently, JEMH is setting both the from: and the reply-to: , it could be changed.

  • *Dynamic from addresses is not yet implemented, you will need this in order to have one Event Listener Project Mapping and differing from addresses that are derived from the from address.* => That would be a perfect solution. I was not even expecting it to be dynamic, but just the option to enter the from-address manually, but not in the Profile or a project mapping for an Event Listener, but e.g. as part of the project mapping for a Profile (overwriting the default from the Profile, if set).

It can be entered, but on a Per Event Listener Project Mapping only, therefore requiring N such mappings if you wanted to do that.  The comment above about dynamically setting the from/reply-to/locale based on what inbound address the mail was sent 'to' applies.

Edge Case: sometimes people BCC JIRA, in such a case there is only a possibility that a delivery address is listed somewhere in the payload.  JEMH has abilities to pickout such values if provided, fallback will of course, be English.

  • So, two options from my point of view: 1. Maybe our customer is willing to pay you to get dynamic from-addresses based on a custom field asap. This combined with auto project mapping based on key123@example.com (and dropping languages) would be the perfect simple solution as we would not need a single project mapping within the Profile, and only one for the Event Listener (with regex .*).

Yes, ability to map Locale to inbound addresses, storage of inbound addresses against issue to 'define' the Locale 'for' that issue (gets fun, if you every wanted to MIX Locales).  This is a feature that would hit the very latest versions of JEMH/JIRA to minimise addition migration effort/cost, thats looking like JIRA 6.4 / JEMH1.7 which already has some major changes.

 

 

Tobias Reibling
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 13, 2014

You shouldn't work on Saturday ;-) Thanks again! Taking into account the number of projects, basic idea really has to be to eliminate the need of having to define *whatever it is* in mappings on a per-project base (Profile or Event Listener). So https://thepluginpeople.atlassian.net/browse/JEMH-1347 (as long as implemented as part of Profile or Event Listener and not as a mapping) combined with what you have described for dynamically controlling the language based on parameters like sender address etc. would be a perfect solution. In all cases, route mails to projects would definitely need to be based on project auto-mapping for projectkey@example.com. I will discuss that with our client next week and would contact you directly in case they are disposed to pay for enhancements or for getting them faster.

0 votes
Andy Brook
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 12, 2014

Chuckle.  OK, big instance, therefore not trivial requirements:

> approx. 1000 (!) different projects

From the start, don't think about project mappings wink think about auto-assigning projects based on the addressee, its automatic and requires no configuration/maintenance other than a checkbox.  It does however not come with frills of per-project customization from Project Mappings, but uses some sane defaults.  If you need Project Mappings you can add them too.  Having addressee based project selection (for create) requires your infra to be 'better', you will need a dedicated mail domain just for JIRA, not share a @yourco.com domain.  By creating jira.myco.net then, all recipients can be routed to projects with keys matching the recipients, without wondering if/but about user matching in the same domain, dont do it.  Configuring one mailbox for many addressees can be done automatically, if you have an Exchange admin who sucks teeth at this, consult a Linux pro, referring Postfix and Dovecot.

> non-jira email users should be able to create issues, comment and receive notifications on create, resolve, comment

Yes, given 1000 project scenario, some compromises need to be made. For example, JEMH can have a single Event Listener Project Mapping, normally this is per project, but you can use a regexp to match the project key, eg .* will match all projects.  I would say, that unless you have incredibly uniform notification schemes, that in this scenario, you'd use JEMH for just email only user notification.  This will then work well with low overhead and complexity - you'll have enough going on managing standard JIRA notification schemes I would think.

> routing of issues created into a project should be done via the recipient email address (individual mail addresses, not related to existing JIRA user, not matching the project key)

Well, thats different to supporting mass issue creation in projects with low overhead, but is what JEMH Project Mappings do.  If you have a 1000 projects, thats a lot of projects.  The existing UI will probably be usable to a few hundred, but 1K is pushing it, and the UI would need to be improved to make the experience better.  Currently, nobody is asking for that.  Matching on sender/or inbound addressee is possible.  

Having many concurrent inbound mailboxes wont scale past 60 or so, its one thing that drove JEMH development 6 years ago.  The project auto-assign feature was the 'wow' it just works feature to solve that.  If you also want to have Project mappings based on inbound addresses, that gets interesting, and overhead will come in terms of managing project mappings and the various addresses.  At a physical level, all those mails need to get routed into a 'pickup' mailbox effectively multiplexing inbound addresses to get them into JIRA into the right project.  Once they are there, standard JIRA mailbox and the default smtp addresses set per-project (if done) would apply.

> when sending notifications to Non-JIRA-users, the from-address needs to be the respective recipient address (users need to get the email back from the same address they had sent it to)

Thats harder.  The 'easy' way above I mentioned about having a single Event Listener mapping with .* can't be used if you want this.  To do this, there needs to be a one-to-one relationship between Event Listener Project Mappings and JIRA Projects.  Then, JEMH can set the from address to be some static address.  At this time I have had feature requests to allow the From: address to be set "from" an issue custom field, which is in this area, but it not implemented currently.

> As we need to route issue creation based on recipient email address, there seems no way to avoid having 1000 project mappings in the profile

If your requirement is the same for all 1000 projects, yes, you're going to need an awful lot of Project Mappings.

> and we probably need project mappings in the Profile anyway as we have to control different languages for notifications via the "proxy user"

Yes, you've noted that the Locale of the 'proxy user' matters for email notifications to email only users.  Whilst I know that is the case, nobody has ever asked for more complexity in this area, eg, Project : Locale or similar.

> One event listener with one single project mapping should be okay, as there are no differences (using a regex to automatically cover all project keys).

As I mentioned, this conflicts with your requirement to set the from: address.

Question1 (well, not really):

For issue creation relates to the custom event for 'technically' complete issue notification

Once created, outbound notificates are through Custom Field > Event Listener Project Mapping / enabled events.

Question 2

> What would be the correct way to replace the default JIRA outgoing mail from-address

JEMH can current set ONE address as the from address in a JEMH Event Listener Project Mapping, NOT what the original email was sent to.  Dynamiclly setting the from address from an issue custom field / other source is an open feature request: https://thepluginpeople.atlassian.net/browse/JEMH-1347

> this would only be possible using a Project Mapping for each project in the event listener and setting the from address in this mapping - resulting in another 1000 project mappings, but this time for the event listener! Is there any way to avoid that?

No, you need a dynamic from address, to be able to retain a simple regexp based JEMH Event Listener, issue above.

Question 3

> (is) project routing for issues created based on recipient email is a very bad idea in this case and really needs to be replaced by routing based on project key in email subject 

Well, the email addressee would be the easiest way to get mail TO a project.  Once there, subject based issue keys are there to resolve issues in replies.  IF you want to have custom issue creation (ie you have wildly different requirements in terms of initial issue type, priority, assignee, components etc etc) then you will need individual project mappings.  The flexibility is there, inherently there is nothing 'wrong' with lots of project mappings but there will be a calculation cost for 1000 Project Mappings with more than one simple address based rule.

> (which eliminates the from-address problem and could make project mappings obsolete, too)?

I dont see how it would, varying from-address is something that has to happen at the point the email is sent (Event Listener Project Mappings != Inbound 'Project Mappings').  

Summary

  • Project auto assign with a dedicated JIRA domain will work well for any number of projects.  
  • To vary how issues are created past Profile defaults requires Profile Project Mappings, one per project.
  • The UI could slow down a bit listing 1000 project mappings in the profile view.  Some higher level abstraction will be needed to make it easier to manage.
  • To Notify email only users can be done with one Event Listener Project Mapping
  • Dynamic from addresses is not yet implemented, you will need this in order to have one Event Listener Project Mapping and differing from addresses that are derived from the from address.

Your time is up, 24hrs min for next reply wink

Andy Brook
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 12, 2014

And yes, I really wish questions would actually *USE* the entire screen width...

Tobias Reibling
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
December 12, 2014

Hi Andy, Thanks a lot for your quick and detailed answer that basically confirms that our requirements are problematic and we need to re-think the concept. Let me just focus on your summary: *Project auto assign with a dedicated JIRA domain will work well for any number of projects.* => I played around with the auto project mapping based on the project key in the addressee part of the recipient email (key123@example.com creates issues in project with key123, ...), which would avoid project mappings. But this does not resolve the problem with the notification language (we might convince the customer to drop this) and also does not help with the from-address problem. As long as external users will have to understand that they have to send mails to key123@example.com or key456@example.com or key 678@example.com, of course it is essential that they get any kind of notifications back from the same address. *To vary how issues are created past Profile defaults requires Profile Project Mappings, one per project.* => All projects are exactly the same as regards all other handling/JEMH configuration, they require no different configuration at all, no assignees, no issue types ... except for the language (if possible) and the email address. *The UI could slow down a bit listing 1000 project mappings in the profile view. Some higher level abstraction will be needed to make it easier to manage.* => I do not expect JEMH UI handling such an amount of mappings well, and agree that usually, nobody needs that (let's see what JIRA says about > 1000 projects ...) *To Notify email only users can be done with one Event Listener Project Mapping* => Yes, clear. *Dynamic from addresses is not yet implemented, you will need this in order to have one Event Listener Project Mapping and differing from addresses that are derived from the from address.* => That would be a perfect solution. I was not even expecting it to be dynamic, but just the option to enter the from-address manually, but not in the Profile or a project mapping for an Event Listener, but e.g. as part of the project mapping for a Profile (overwriting the default from the Profile, if set). So, two options from my point of view: 1. Maybe our customer is willing to pay you to get dynamic from-addresses based on a custom field asap. This combined with auto project mapping based on key123@example.com (and dropping languages) would be the perfect simple solution as we would not need a single project mapping within the Profile, and only one for the Event Listener (with regex .*). 2. We look for a setup that eliminates the need of a dynamic from-address as all emails are sent to support@example.com and all notiifcations are sent from this address - anyway it is a very special concept to work with 1000 different mail addresses (catch all on the level of mailserver, not really 1000 mailboxes or aliases, but with all the ugly consequences of a catch all). That was the idea of mapping from a project key in the subject. Of course this would create a huge amount of routings into the fall-back project as users will simply forget to put the key (but they will also send to the wrong email address, and the catch all will always catch it all). Unfortunately we can't use sender addresses, as many senders are related to many different projects at a time ...

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events