How control the order of Issues Processed when using a Scheduled Trigger

[Doug Levitt]

I have written multiple Scheduled Triggers which are configured to "Run a JQL search and execute actions for each issue in the query".

With my latest Automation, I tried to influence the order in which the Scheduled Trigger "processed" the Jira issues by including the Order By clause in the JQL search.

However, it seems that the Scheduled Triggers process the incoming Jira issues in an nondeterministic order.

Is there a way to define a Scheduled Trigger to control the order in which the Jira issues are actually processed?

[Mykenna Cepek · Accepted solution]

My guess is that the parallelism that Automation uses to handle looping on multiple issues is causing the non-determinism that you're seeing.

That was a performance / consistency tradeoff that Atlassian made, and it prevents a lot of rules from behaving as one might otherwise expect.

_ _ _ _ _ _ _

I had a crazy idea for your use-case which might lead to a solution (maybe even a new workaround/approach for this kind of scalability limitation, for us rule authors)...

What if you had RuleA which iterated through your projects, and it triggered RuleB using a webhook containing the issues of interest in just that project?

RuleB would receive the list of issues, process them, and output the appropriate slack message. Note that this would be a per-project approach that would not work to aggregate all project data into a single report or message.

Since this webhook triggering can only pass a list of issues, RuleB would likely have to be able to look up all project-specific information (e.g. Project Name, or target Slack channel for that Project) from any given issue. Lots of details to resolve there.

Just a thought for a radical approach.

[Doug Levitt]

Thanks @Mykenna Cepek 

Your "crazy idea" sounds quite intriguing.  I need to think about this a bit.

As a side note, I was also think about ways to pass data between automations using user entity properties.  Though, I wanted to set the key dynamically (say based upon a variable).

Thanks,

Doug

[Doug Levitt]

@Mykenna Cepek 

I thought about your proposed solution.  It's not so crazy.  I think I can use this to solve my other question.  Thanks so much!

[Doug Levitt]

@Mykenna Cepek 

> I had a crazy idea for your use-case which might lead to a solution...

I actually implemented this idea to visualize dependencies (issues that are blocking one of our Squads, based upon an "is blocked by" link).  I created the "administrivia" project and created a separate ticket for each Squad (issue type = "Squad").  I created a whole bunch of custom fields to track (for each Squad) metrics - cycle time, planned vs committed, velocity, health of backlog (by counting the # of ready future sprints, based upon velocity).

So, I now have a scheduled automation which -- on a daily basis -- iterates through all Squads w/in the "administrivia" project and for each Squad invokes a webhook (passing in the issue information associated with that Squad, including its name).

Then, I have an automation that listens on that webhook and processes all issues that the Squad works on.  Then, it goes through all open issues, examining the links and populates fields per issue (total # of dependencies, # of open dependencies, list of dependencies - issue #, status, summary, etc.).  It works surprisingly well.

As a note, the "administrivia" project also contains a history of all Sprints (issue type = "Sprint") -- with metrics per Sprint per Squad (that gets rolled up to each Squad).

So, it took a while to figure out how to do this, but it is very intriguing the possibilities.

As an example, I now have a dashboard for the active sprint (we have ~ 30 Squads having parallel/synced sprints) where I can see each Squad's Sprint Goal, Start Count of PBIs, Start Sum of SPs, Current Count of Open PBIs, etc.  I have another dashboard that shows all future Sprints. 

Thank you for making this suggestion.  Your perspective provided a entirely different way of looking at data management w/in jira.

[Mykenna Cepek]

Thanks for circling back, @Doug Levitt, and sharing what you learned and how it is all working for you -- very cool!

I take the time to swing at various pitches here in the community forum. It's very nice to get this kind of feedback when it's a hit.

[Bill Sheboy]

Hi @Doug Levitt 

Reading over your question and the response posts, I am wondering about the "processing" which this rule does.  And what the need is, other than reporting in a specific order when completed.  If only reporting order is needed, perhaps let issues "process" as they will, in any order, and then use a dashboard or subscribed filters for reporting.

However, if there is truly a need to process issues in a specific 1-N order, I can only think of two ways to try this.

• Using the REST API to gather issues, build your own custom application to process and update the issues
• Process in automation rules, one issue at a time...very, very slowly...using a slow repeat scheduled trigger and lookup issues to force only processing one issue per rule run.  Such a solution seems like a bad idea as it could create a collision, processing the same issue twice due to performance and automation outage issues.

Kind regards,
Bill

[Doug Levitt]

Thanks @Bill Sheboy 

Probably, I need to think of another solution (to my current problem).  It's pretty clear that what I was trying to do is not currently possible.  Thanks!

[Curt Holley]

I agree with @Mykenna Cepek  and @Bill Sheboy  and think you are going to struggle to take full advantage of the tool if you don't go beyond "general" rules.
Hard to know how to advise further without more detail on the purpose of the rule.

I'm supporting 1400 users, hundreds of projects and teams. Use many global and multi project rules and have hundreds of automation rules all up. No-one is finding it an overhead in comparison to the overhead the rules save. 
Which is it in a nutshell, don't create rules that are more work/hassle than the problem the rule is out to solve....everything else, is winning.

[Doug Levitt]

Thanks @Curt Holley 

Your advice is quite sound.  Appreciate it.  It's back to the drawing board :)  

[Curt Holley]

Hi @Doug Levitt 

Interesting question. If ORDER By doesn't do it, I'm not sure it can be done.

Question: Why does it matter? If you need it to do certain issues prior to others, I would recommend breaking the rule out into 2 (or how ever many you need) and schedule accordingly.

[Doug Levitt]

@Curt Holley 

My automation traverses through 100s of issues owned by different Teams and then outputs the results into a "report" (which could be an email message or a slack message).  I want to organize the output by team.

We have about 30 teams, and obviously I don't want to create a different automation for team (as it doesn't scale from a supportability perspective).

Please note, the above is simply a single use case.  I have others.  The entire goal of automation (IMO) is to address deficiencies in the base Atlassian product, as it doesn't really work well at scale.

[Curt Holley]

Question: Are you using the Advanced Roadmaps (AR) based Team field (from the Shared teams section of Plans) or just a custom field of your own making?

As the AR Team field isn't a sortable field (currently).

Another consideration if using the "Look-up issues" action is, it is limited to 100 issues per look-up. Which might mean the per team option does make sense.

P.S. not sure what you mean by:

We have about 30 teams, and obviously I don't want to create a different automation for team (as it doesn't scale from a supportability perspective). 

Why doesn't it? Simply create one, copy it and change the Team in the new one, save with the team name in the rule title, rinse and repeat.

[Doug Levitt]

@Curt Holley - I am not using Advanced Roadmap's Team field in the Order By clause (because, as you said, Atlassian has only partially implemented the Team field).

So far, I have tried using both a Custom Field and the Sprint Field in the Order By clause.  Neither works (as the Automation returns the Jira issues in what appears to be a random order).

As you mentioned, the Lookup Issue is an interesting tool, but it has multiple limitations:

- It is limited to 100 issues

- The list of issue properties returned is quite limited (e.g. it doesn't include custom fields or Sprint)

I have used it for other purposes, but it is too limited in this case.

Why doesn't it? Simply create one, copy it and change the Team in the new one, save with the team name in the rule title, rinse and repeat.

While that works, it's very brittle and costly to support.

As an example, imagine I want to make a slight tweak to the biz logic.  Am I going to have to make this change to 30 Automations.  And then test each one (as it is so easy to make a mistake when using the editor).

I have done my best with our 30+ Team implementation (supporting ~ 200 users) to avoiding hard-coding by creating "general" automations.

I have run into some limitations with this general approach.  As there are cases, when 30+ teams return too many issues.  So, with some Automations, I have split the 30+ Teams into "processing groups".  In those cases, only the JQL Query is hard-coded.  Everything else remains general. 

Thanks.

Doug

[Mykenna Cepek]

I'll take issue with the suggestion that:

The entire goal of automation (IMO) is to address deficiencies in the base Atlassian product, as it doesn't really work well at scale.

While I'm quite outspoken on the limitations of Automation in Jira, I find myself coming to Atlassian's defense here.

The goal of the product (I believe) was for Atlassian to offer a no-code approach to handling some complex Jira use-cases otherwise not supported in the base product. Code Barrel had a good solution, and Atlassian assimilated them. That's how we got here.

There are lots of other approaches to handling complex Jira use-cases. Using the API was mentioned here. There are several other (paid) Apps for Jira which support different and good approaches to automating things. However, I believe all those require actual coding (e.g. in Groovy or Java or Python or whatever).

I'm a former software developer, so "having to write code" doesn't worry me. In my second career as an Agilist and Atlassian Admin, however, what worries me is the "legacy of complexity" that I use to solve problems for my employer.

I loved writing code in Groovy. And I've explicitly chosen (at multiple employers) to not go there. Why? The next assistant they hire to help me (for example) is unlikely to know Groovy, or even how to code. Or when I'm hit by a bus, who can fix the complexity that I've left behind?

Can my boss, or peer, fix what I've done when it breaks (because I'm on PTO, or sick)? The high value in the existing Automation for Jira is the ease of use.

Trust me, I'm frustrated too. I've kept expecting Automation to behave "like a real programming language". It kind of does, and kind of doesn't. Big sigh.

When we hit this wall, and Automation just can't do it, there are really just a few constructive options:

• Adjust the goal to be within the capabilities of Automation.
• Look for another approach (API, or App).
• Brainstorm for new ideas and collaborate here in the community.
• Give up.
• Add, or vote for, the needed Feature Request at JAC (and then still pick one of the above).

Those of us who have progressed well beyond the basics of Automation need to accept its limitations as well. I think this is true for any tool; eventually, as an expert, we will hit the wall.

[Curt Holley]

Exactly @Mykenna Cepek 

[Doug Levitt]

@Mykenna Cepek 

Thanks for your response.  I do understand the coding option.  However, it's not a viable option for us, given the concern about long-term maintainability.