Need to think of a workaround for Jira Automation AND/OR ScriptRunner Limits

[Zachary Howard]

Some background: Jira Cloud, Premium License. Our company uses a custom number field called Internal Priority. This field is used by management to determine which Epics are the top priority. We have a ton of these Priority statuses, each with their own meaning, so we cannot use the built-in Priority field. 

We have a prioritization meeting every week where management will update these numbers, and those updates will then get pushed to Jira in bulk. Our teams need the internal Priority number on each child issue because they view and sort the child issues on a Kanban board that does not show the Epics. 

We had built an automation that will update all child issues Internal Priority field when the Parent Epic's Internal Priority field is updated. However, because we update over 100 of these Epics after every prioritization meeting, and each Epic can have between 5 to 80 children, we end up hitting the runtime limits in Jira Automations.

Attempted Solutions:

1. More Automation Rules: We have already tried breaking our automation rule down into multiple rules based on the Internal Priority number, however the problem is that most of the Epics fall into one or two of these and those rules still hit the automation limits. 
2. ScriptRunner Listener: We had purchased ScriptRunner, thinking that it would help us to bypass this limit, but we ended up running into another limitation thanks to Jira API Rate Limiting. This is because the trigger for ScriptRunner is set to when the Epic is updated. This causes all updated Epics to trigger at the same time, which pushes too many API calls (to update the child issues) at once. I even programmed in a wait time to compensate for this and it still ends up failing. We cannot split this automation in ScriptRunner because the limit is for the API, and not the individual rule like in Jira Automations.
3. ScriptRunner Scripted Fields: I thought this might be a viable solution, however there is another limitation here where the Scripted Field will only update after the issue is viewed in Jira by a user. Since we need to use this field for reporting and filtering using a Scripted Field won’t work.

Any suggestions or tips that we could look into to solve this problem?

[Bill Sheboy]

Hi @Zachary Howard 

However you solve this, if the Epics are updated during teams' working hours, I suspect there is already a visibility problem: because of the time needed to update child issues, priorities will shift while teams review them on the Kanban boards.  I suspect a process change may help more than trying to solve this with automation rules.

I hypothesize you are encountering two limits with the automation rule approach you describe:

• With so many Epics updated quickly, that triggers multiple instances of the same rule, and that hits the upper limit of "Number of rules that can run at the same time"
• Because of that, the continual processing to catch-up exceeds the "Daily processing time" limit, resulting in a THROTTLED error, halting rule executions for the entire site.  (This is in addition to any other automation rules trying to run for other reasons.)

As you have already tried breaking up the execution with multiple rules, that likely ran into the same situation.  An alternative would be to use multiple rules, on staggered scheduled triggers with JQL to limit scope, and checking using an additional custom field (e.g., Internal Priority Update DateTime) on the Epics to limit execution to "today".  That will certainly be difficult to tune and will still fail if there is any automation outage that delays execution on the schedules.

As a process change alternative, have you considered re-ordering (i.e. ranking) the Epics in response to the change to your Internal Priority?  If that was done, the Kanban boards could use swim lanes based on Epics to order the child issues, assuming the board filters use an ORDER BY Rank ASC clause.  This approach eliminates the need to cascade the field value down to child issues.

Kind regards,
Bill

[Sydney Marsing]

Hello Bill! Co-worker of Zachary's here, thank you for such a detailed response.

RE: Hitting the Limitations
Our working solution right now has been to use multiple rules that do trigger at the same time but have much more limited scopes:

WHEN Issue's Internal Priority field is changed
IF the Trigger Issue has Children
AND IF the new Internal Priority value is between X and Y
...

Each automation has different X and Y values, with scopes that start very narrow and broaden near the bottom of the priority list. (0-5, then 6-15, then 16-30, 31-50, 51-100, 101-200, etc.) This makes it so the automations handling the highest priority updates are responsible for the fewest Epics--making them the least likely to fail and the fastest to update.

But, as we've grown, we've had to add more and more of these automations with smaller and smaller ranges, and we're worried about how sustainable this is. It might start eating into our automation budget, and the more near-identical automations we have, the higher the effort and risk of user error we have when maintaining or updating them. 

I've considered using JQL filters, but I've been cautious about them so far, since we may start modifying around 10,000 issues per update. I'm not sure how to set the JQL filters narrow enough to avoid the risk of exceeding the search limit--and if I did, we might hit the same problem of having so many automations doing the same thing that it becomes difficult for the admins to revise or maintain. Definitely open to advise or suggestions on this point.

[Bill Sheboy]

Hi @Sydney Marsing -- Welcome to the Atlassian Community!

Even with the divide-and-conquer approach of multiple rules, I believe the problem still exists because of volume of changes potentially leads to throttling of rules.

How often are those Epics re-scored?  And, do they all change each time or only a subset of them?  Let's assume it happens once per week.  In that case, rules with Scheduled Triggers and JQL could operate on a subset of the issues (selecting ones that changed since the last run and for specific priority ranges), and the schedules could be staggered to reduce the load.

Did you note my suggestion about not updating the child issues, and instead relying upon the Rank field and swim lanes to solve this?

[Sydney Marsing]

RE: Updating the Epics and not the Children

I think you are right that using Kanban Boards that filter and sort by Epic rather than the issues themselves is probably the simplest and most sustainable option long term. Many of our teams that work cross-project already have their boards set up this way. 

However, there are a few benefits to having the Internal Priority on every card that we're still trying to account for, before we switch to using only that solution.

1. Internal Priority as a visible field
   There is no setting that would display the swimlane Epic's internal priority on the Board itself. Currently, teams get around this by setting Internal Priority as a visible field on the issues themselves, or adding it as a default column in their dashboard filters. 
   This lets the team follow more complex directions from the lead without having to open and scan through multiple cards. (i.e. "Priority 5 is due sooner than priority 1, so alternate between them", "Switch to helping another team once the highest priority work in your own queue is 15.")
2. Security
   We have had problems with Project Managers trying to raise the priority of their own projects without permission and causing confusion. To bypass this, we've removed the Internal Priority from all of the Edit/View screens, so they can be seen in columns or set as visible fields on Boards, but not directly edited without using bulk updates or admin-restricted transitions. However, if the Internal Priority is not found on the issues themselves, and also can't be looked up by clicking the issue's parent link, the difficulty of checking the priority of an individual issue gets much higher. 
3. Basic JQL filtering
   We currently have only 1 employee who's comfortable using Enhanced Search with ScriptRunner. Having the Internal Priority on the Child cards allows everyone else to run simple queries like "List the child issues of the 10 highest-priority Epics that are are yet to reach X status" or "Show me all the currently impeded issues, sorted by Internal Priority."

[Bill Sheboy]

Yes, I completely understand those concerns, and...

For the field priority issue, although I do not always recommend it, a workaround is to concatenate the field value to the Epic's summary so it appears in views.  This can be brittle and may require extra Epic updates, but could be done with an automation rule.

For the security issue, that "smells" like there is a root cause problem to address: why are some project managers changing the priority rather than using the defined, organization process for prioritization?  Investigating that may reveal education and improvement opportunities for the process.

For the JQL filtering, for those not comfortable with writing such queries with ScriptRunner, could you save some as template queries which they may use directly or add to their own?  Those could be initiated from the filter list, a dashboard, or manually:

filter=12345 AND my_JQL_goes_here

[Trudy Claspill]

Hello @Zachary Howard 

What specific Automation limits are you hitting, and what automation rules have you configured when hitting those limits?

[Sydney Marsing]

Hello Trudy!

I think Zach would be able to speak better to what we've tried in ScriptRunner, but in Jira Automations our rules have worked like this: 

WHEN an issue's Internal Priority field is updated, 
IF the Trigger Issue has children
AND IF the new Internal Priority is between X and Y

Then FOR that issue's children
IF their Internal Priority does not match their Parent's
THEN copy their Internal Priority from their parent 

The error we have mainly hit is the timeout error for having an automation exceed 3000 seconds in a 12-hour time period. So far, our solution has been to split the automations that fail in two: One for Internal Priority between X and Z, and another for Priorities between Z and Y.

So far this has worked, but I'm concerned about "bloat" if we keep doing this forever: are we missing some obvious way to handle this task more efficiently or with fewer moving parts to maintain?