Automation order of operations?

[Stephen Lee]

Hi,

I'm trying to make an Automation rule for the case of transitioning a parent story (with children) to Done. I would like it to:

1. If there are any subtasks that aren't in certain statuses, transition those to Done

--then--

2. If there are any subtasks that aren't Done at this point, move the Parent story back to In Progress (or otherwise prevent it from being transitioned to Done)

Here's what I have. In testing, it seemed inconsistent: I saw once or twice where it successfully moved the children to Done, then allowed the parent to stay Done. But then another time I saw it transition the children to Done, and then move the parent back to In Progress.

Is there a race condition between these top-level directives? Or are they supposed to execute / complete sequentially?

Thanks

[Darryl Lee · Accepted solution]

Hey Stephen - yes, you're definitely running into an order of operations thing. Namely: AUT-517 - Rule branch order is non-deterministic. In some cases it would be better for branches to execute in order.

But I'm also a little confused by your rule.

You want to set all Sub-tasks to Done, but if any were not Done (prior to setting them?) then the parent should be kicked back to In Progress?

That's ... interesting. As my old manager would say -- what's the business case for this?

Because don't the two things kind of contradict? Either all subtasks are Done, and the parent can stay Done, or some subtasks need to completed, and the parent should be kicked back to In Progress.

I don't understand the purpose of automatically moving subtasks to done.

[Stephen Lee]

Thanks Darryl!

So, I only want to automatically set some subtasks to done - any that are in the latter half of our available statuses. It would have been more clear if I wrote it that way, but like if they're Merged, Ready for Test, In Test... then at that point I'd feel confident automatically transitioning them to Done.

But if any were To Do, In Progress, or Blocked, then I wouldn't want to auto-transition those to Done, and would want to make sure they don't get missed. And in that case I also don't want the Parent story to stay in Done, bc if so, those children sort of get buried, since they aren't visible on the Board as standalone issues.

I should also mention that we use subtasks for developers breaking down a story into smaller tasks, as well as any bugs filed against the main story during the course of the sprint.

Is there a better way of solving this? 

[Darryl Lee]

Ah, ok that makes more sense now. Thanks.

But as I tried to come up with a solution, I hit the same brick wall - rules don't run in order.

@Mykenna Cepek wrote a great rant about this.

I guess you could do something hacky like:

Rule 1: Triggered off of parent moving to Done: Automatically move "Merged", "Ready for Test", "In Test" subtasks to Done and also write a comment back to the Parent saying "Approved Subtasks automatically moved to Done"

Rule 2: Triggers off of Issue commented: if last comment ( {{triggerIssue.comment.last.body}} ) = "Approved Subtasks automatically moved to Done", then do the check that there aren't any other subtasks that are in the "Unapproved" state and if so, kick the Parent back to In Progress.

In rule #2 you'd have to make sure to check the Allow rule trigger checkbox:

Check to allow other rule actions to trigger this rule. Only enable this if you need this rule to execute in response to another rule.

Any other ideas, @Mykenna Cepek or @William Sheboy ? I was almost going to suggest issue entity properties, but there's no trigger for those...

[Mykenna Cepek]

Looking at the screenshot of your rule in the original post, it seems you're close, and that maybe a different concern is causing the inconsistency you see.

Have you tried simply putting a "Refresh issue data" action before this conditional:

It occurs to me that caching might be foiling your logic, and the refresh might bring the changes made in the previous looping/branch back into sync with the conditional that scans the sub-tasks again.

If that doesn't solve the problem, I'd go to another step to ensure that I'm understanding exactly what's going on during the rule execution. And that's simply to litter the rule with tactical debugging, in the form of "Log action" components. A few examples:

Rule triggered on {{issue.issueType.name}} {{issue.key}} with Account='{{issue.account.name}}' and change event: '{{eventType}}'

This, at the beginning of a rule, helps ensure I understand the initial context. Super helpful when sifting through the Audit Log, to remember which test was which.

Found {{issue.issueType.name}} {{issue.key}} with Status="{{issue.status}}" and changing it to Done.

Putting this after an action that actually updates an issue status does two things: it confirms that the "Edit issue" action executed (since the log action is after it), and it verifies the edited issue before and (assumed) after state.

After first loop.

Something simple like this confirms where I am in the rule. Just a sanity check "did it get to this point?"

A WARNING: The order of messages in the audit log also tends to be non-deterministic. From being a developer I know that some logging frameworks are just that way by design (decoupling potentially multithreaded applications from a necessarily single-threaded "write to log" component - you don't want log messages stomping on each other). So pay very little attention to the ORDER of what you see in the audit log, and focus on the presence/absence and content of the log messages.

Would love to hear about what you learn next!

[Bill Sheboy]

Hi @Stephen Lee 

@Darryl Lee is correct about the asynchronous execution of parts of the rule...Specifically there is no guarantee the branch is done before the rest of the rule.

Darryl's suggestion seems like one way to do this: split this into 2 rules, where the first rule's completion sets a condition that can trigger a new rule.

The other way is to change your second condition in your rule, such as this:

• Trigger: issue transitioned to done
• Branch: on sub-tasks
  • Condition: sub-task status not in (To Do, In Progress, Blocked)
  • Action: transition to done
  • Action: add comment
• Condition: if some sub-tasks status not in (To Do, In Progress, Blocked, Done) ...or... status in (any other existing status in your flow)
  • Action: transition to In Progress
  • Action: send email

Changing the second test to include everything you already caught in the branch...but which may have not yet processed, and will prevent the race condition.

Best regards,

Bill

[Stephen Lee]

Thanks for all the timely and thoughtful responses everyone!

Mykenna - I appreciate those debugging / logging tips, which will be useful in any Automation going forward!

Bill - Your last suggestion was a total "duh" moment :p that totally makes sense, and I'll give it a try now! 

[Bill Sheboy]

When stuck getting a rule to work (in Jira) I use:

• Consider if I can invert conditions or make them mutually exclusive so the parts of the rule can run independently and in parallel
• When sequential processing is needed instead, multiple rules and semaphore techniques are key
• Worst case solution for race-conditions with parent/children, use re-fetches to slow things down and reload the issue state

__Bill

[Stephen Lee]

That worked perfectly! Here's my final config, for posterity. Thanks again!

[Darryl Lee]

@Bill Sheboy deserves the credit for this answer. I'll shoot you some Kudos, Bill. Thanks for the assist!

I always have such a hard time wrapping my head around inverting conditions, but you're right, it's often the way. (See also: Jira Expressions. Oooh, the boolean logic that @David Fischer has had to school me on so often.)

I also appreciate @Mykenna Cepek's words of wisdom. I'm decidedly not a developer, so it's great to have that deeper perspective.

[Moheeb Qupty]

Hi Guys,

I know this a little old post, but I just saw and it was worse adding in my experience and our solution that is maybe a bit more complicated but I might have a solution.

In our case we have an Epics that are requirements and those Epics have a link "Depend on" to Stories (also those Epics might children, child to, other Epics/parents but let's leave that for a moment).

Our goal to synchronize the Fix Version of these Stories with their Epics, in a way that when we change Story's Fix Version the Fix Version should propagate to all the related Epic. Note that we don't add the new Version to the Epics, we need to replace it in the list of Fix Versions for the each Epic (each Epic has all the Fix Version of their stories and of course Versions can be shared): for example if in a Story we deleted the Fix Version, this Version should be deleted from the Epics unless it's used in other Stories of course... etc. So the solution is to reset the Epic's version and rebuild/collect the versions from the Stories starting with the "new" changed Story.

The way I implemented the solution is Rule #1 when a Fix Version is changed in a Story, it branch out for the Epics and for each one reset the Fix Version of the Epic with only that particular changed Version (not adding).

Then another Rule #2 triggered when the Fix Version is changed for the Epic, and in this Rule the Epic branch out for each of its Stories (in a condition that it's not that same Story that was changed in first place, because the Epic already has the Version) and reset coms custom field in the Story and change it with the trigger issue key (Epic key #).

Rule #3 is triggered when that custom field is changed, in a condition it's not empty it will branch out with JQL on that custom field and fetch the Epic to "add" the trigger issue (Story) Fix Version to the Epic's Fix Version.

This work perfectly in many cases, until we test it in some hierarchies when we have same Stories are linked to different Epics and the race and order of things ruin the final resutl.

Since branching out from the Epic to their linked Stories to update the custom field happens on the same time (we checked on the log and comments), so we have rules that executed in parallel and they update the same custom field of the same Story!

I think this cause issues to update the list of versions correctly into the Epics. So we have consistencies and sometimes although we see from the comments that the Epic # is retrieved and the correct Story's version should be added, IT'S NOT ADDED and the Epic missing 1 or 2 versions.

I hope I was clear in my explanation, but if not clear I can clear and send screenshots from my rules.

The way I am thinking to solve this it to add safe-guards when executing the branching so we have some order on rule executions (Rule #3) when executed on the same Story:

Rule Execution_1 of Epic_1->Story_X and Rule Execution_2 of Epic_2->Story X is run in parallel, better that Rule_Execution_2 delayed or executed after Rule_Execution_1.

This is a limitation because of the solution I chose to use custom field in a Story, I couldn't find other solution to store some info and re-use and also can trigger a rule.

[Mykenna Cepek]

One workaround I can see, given the capabilities of Automation today, is to switch your ruleset from being trigger-based to being timing-based. The downside is that the fields you are maintaining (e.g. FixVersion) will not be updated ASAP, but merely "soon". For example:

• Rule 1 triggers every 15 minutes to scan for Stories with changed FixVersions since the rule last ran. It updates the FixVersions in Epics as needed, then quits.
• Rule 2 triggers 5 minutes after Rule 1, and does its thing (finding Epics changed since it last ran, and updating child Stories).
• Rule 3 triggers 5 minutes after Rule 2, and does its job.

So within 15 minutes, all your edits are complete for any given change.

Advantages of this approach is that you have control over preventing the rules from running simultaneously, and the results should be much more predictable.

Some substantial assumptions for this to work:

• You are on the Jira Premium or Enterprise plans (so that you don't hit the limits of Jira Automation rule executions per month).
• All three rules use the Scheduled Trigger based on a cron job (to ensure even spacing throughout a given hour).
• Each rule executes in 5 minutes or less.

That last one is tricky, but you can watch your rule execution times in the Audit Log. If necessary, you can increase from 15 minute to 20 minute cycles (or longer).

(Bonus points: have a rule time itself, and notify you if it exceeds it's "time box")

A core assumption in this approach is that by running frequently, there will be less work to do each run (and so each rule will complete more quickly). However, if you have other rules that might run and create a lot of work for Rule 1 (for example), then you'll have to tune this appropriately.

[Mykenna Cepek]

You definitely have a complicated use-case, with 3 rules potentially running at the same time, potentially reading and writing data from/to the same issues. This is a known problem (IMHO) with Automation, and one that is not easily solved.

Note that allowing rules to trigger when an issue changes is a great feature, but a subtle assumption is that rules execute quickly (kind of like an interrupt routine in operating system code is expected to complete its execution very quickly).

Full disclosure: I too have rules that sometimes run for many, many minutes.

One of the amazing things about Automation in Jira is that it can be used for these broad use-cases -- attempting to keep data synchronized across dozens or even hundreds of issues. And when all the logic is in a single rule, that can work.

Another disclosure: I've engineered rules so large that I hit the "maximum number of components in a rule" limitation. That forces one to split rules, which then wades into this whole messy area.

Note that certain triggered rules have an important related feature: a checkbox for whether they can be triggered based on changes made by another rule. Unfortunately, it is too blunt of a tool for this problem. We actually WANT the changes made by one rule to trigger another rule. The problem is we don't want the rules to execute simultaneously.

But that's a tough problem for Automation to solve. Postponing a triggered rule (until the triggering rule completes) would require queueing up triggers, possibly for multiple rules -- and (as a former developer) that's starting to sound not feasible.

However, Automation leads many rule authors to these complex scenarios. Between non-deterministic execution when looping within a rule, and the non-determinism when multiple rules are executing -- we are rightly frustrated.

I've pondered "rule chaining", "fixed delays", and other simple solutions. But nothing seems worthy of even a formal Suggestion, since they are band-aids, not solutions.

I challenge the Atlassian Automation architects to think about these advanced use cases, and come up with workable solutions to help us.

In many cases, the only alternative we have is to tell our users that "Jira can't do that" (sometimes after spending a lot of time trying to make it work).

[Moheeb Qupty]

Thanks Mykenna.

Interesting approach and yep scheduled rules are power tool if used correctly.

The point here to write a JQL that "scan for Stories with changed FixVersions since the rule last ran"  I need to store then "previous" value in a issue property or custom field so in JQL is compare Fix Version with that value.

Yes it's complicated and the deeper the hierarchy the more it's complicated - I have 2 fields/property that I need to safeguard from parallel assignment when the rule execution run on the same issue (Epic or Story).

Possible solutions for the Atlassian architects that they can add to the Automation:

1. Give us action to delay the execution for given time based on our input (now we rely on re-fetch for fix seconds 0, 0.5 , 1, 5, so each execution I can give a different delay seconds based on some logic in that way I force the branching (rules trigger rules) rules to run in different timespans.

2. Add action to wait or put itself in idle mode until some value changed or trigger (if "value" is 1 mean other rule on the same issue is running, so other rules on the same issue must wait until the "value" become 0 or something). Maybe this work best with delays as well as in #1 (This is the way we handle multitasking in coding

Beside rule execution for "realtime" changes, we also have a code (Python-Jira) that run every once a while so sync-up all the issues with their depends, parents and children. Of course in code is much easier to handle this.

I can suggest to Atlassian to think about the above solutions to add it to the Rule Automation tools, definitely they can make it better to handle these situations. Of course they can also add checkbox in the "Branch rule" to be executed in a order/sequence (not in parallel: only when one iteration is done then go to the next iteration (I know they will say this can cause performance issues) .

Anyway beside your suggestion of a schedule rules, I am trying to build a synchronization mechanism based on project level property that works as a queue (I will check how to implement that - project entity/property as a list) , for branch rule will add the issuekey and {{now}} in queue (list of the issues, rules to be executed) then once update rule trigger (when change version on of the issues)  before the actual update will check if the current time {{now}} is less than 1 second (anything less than 1 sec considered parallel for me) of other issues (stored in a the queue list) then I add a 1 or 5 seconds delay. Of course once rule update finish will remove the issue from the queue - of course finetuning needed once I start implementing but I think the basic idea has a chance to work smoothly. 

What do you think?

[Mykenna Cepek]

The point here to write a JQL that "scan for Stories with changed FixVersions since the rule last ran"  I need to store then "previous" value in a issue property or custom field so in JQL is compare Fix Version with that value.

Actually, no. Look for this in your "Branch rule/related issues" component:

Enabling that will ensure that the looping automatically limits itself in that way.
 

now we rely on re-fetch for fix seconds 0, 0.5 , 1, 5, so each execution I can give a different delay seconds

Beware using "Refetch issue data", since it can greatly increase the amount of time it takes for a rule to run. It's not a passive delay but a very active one. I've had rules end up being shut down ("throttled") due to that. It's also not guaranteed to solve the "multiple rules running at the same time" problem.

You might also want to vote for this issue (and maybe add a comment that your use case is for a much shorter delay -- 0.5 to 5 seconds):

• JIRAAUTOSERVER-23

Add action to wait or put itself in idle mode until some value changed or trigger

Well, triggered automation rules are intended to work like this - trigger on change. Having a rule pause midway means keeping a lot of state around -- but that will go stale quickly, causing even more problems. I wouldn't wait for something like that.

we also have a code (Python-Jira) that run every once a while so sync-up all the issues

I'm glad the Jira REST API is on your radar. I didn't want to complicate my previous answers with yet another completely different technology approach. But that can be effective too. Note, however, that it still has the same problems with simultaneous execution!

they can also add checkbox in the "Branch rule" to be executed in a order/sequence (not in parallel

In working with Atlassian Support, I've also let the Automation team know why that kind of "linear execution" mode would be helpful in rules (at the expense of the efficiency that parallelism provides). Let ME make that trade-off, right?

I am trying to build a synchronization mechanism based on project level property that works as a queue

I love the concept. But I'm concerned that two known limitations in Automation will make it difficult (or impossible) for that:

• Variables don't work in "Branch/related" looping, so you can't accumulate a list of things in a variable.
• The way that looping executes in multiple threads in a rule is likely to cause problems while trying to read/write properties and accumulate a list.

I hope I'm wrong, and you figure out a good way to succeed at your goal!

[Moheeb Qupty]

Actually, no. Look for this in your "Branch rule/related issues" component:

Enabling that will ensure that the looping automatically limits itself in that way

Thanks, sure I saw it before, not sure how I missed it:)

Add action to wait or put itself in idle mode until some value changed or trigger

Well, triggered automation rules are intended to work like this - trigger on change. Having a rule pause midway means keeping a lot of state around -- but that will go stale quickly, causing even more problems. I wouldn't wait for something like that.

Actually it can cause pauses/delays not more than your suggestion of scheduled rules. I mean this should be used very carefully only on specific cases when I have a custom field or property in issue level and I want to add somekind of "lock" to prevent update on the same time and give enough time each update follows through  (this applicable only for same issue same field so rule with other issues run smoothly). I mentioned before when we have 1 issues linked to 2 different parents, or updating/adding version on the same parent coming from its children, so I use "add" the fixVersion from multiple children but it seems for the rules that run concurrently it causes issues when adding and some children's version are missed in the parent - so I don't care if some child rule pause a bit before adding its version to the parent so insure the addition works.

we also have a code (Python-Jira) that run every once a while so sync-up all the issues

I'm glad the Jira REST API is on your radar. I didn't want to complicate my previous answers with yet another completely different technology approach. But that can be effective too. Note, however, that it still has the same problems with simultaneous execution!

Actually no if you don't it to be simultaneous it's not in Python code, just loop over the linked children in a row and fetch their respective fixVersion and then add them to the parent... etc. beside that even if we have some concurrent processes we can control it using usual code (adding block/unblock and semaphores not limited to the rule automation mechanism.

they can also add checkbox in the "Branch rule" to be executed in a order/sequence (not in parallel

In working with Atlassian Support, I've also let the Automation team know why that kind of "linear execution" mode would be helpful in rules (at the expense of the efficiency that parallelism provides). Let ME make that trade-off, right?

Bingo

I am trying to build a synchronization mechanism based on project level property that works as a queue

I love the concept. But I'm concerned that two known limitations in Automation will make it difficult (or impossible) for that:

• Variables don't work in "Branch/related" looping, so you can't accumulate a list of things in a variable.
• The way that looping executes in multiple threads in a rule is likely to cause problems while trying to read/write properties and accumulate a list.

I hope I'm wrong, and you figure out a good way to succeed at your goal!

In my specific implementation I can overcome the 1st point about updating the variable/property but for 2nd point I am not sure, yes I need find a way and test.

Will update you:)

[Mykenna Cepek]

I mean this should be used very carefully only on specific cases when I have a custom field or property in issue level and I want to add somekind of "lock" to prevent update on the same time and give enough time each update follows through

My understanding of "Refetch issue data" is that it only re-reads the data for the current issue. There isn't any lock or semaphore action assumed.

It certainly will "add a bit of delay" to the rule, which in some cases might seem like a rule suffering from a race condition is working better. But I am very doubtful that it is a real solution.

[Moheeb Qupty]

Correct "Refetch issue data" only re-reads and add a delay, no lock or anything. In this context I mean if Atlassian added this "waiting", sitting idle until... mechanism.  

And for lock/semaphore I/we implement in the rules, like a flag or property could be in project level or issue level to protect same issue updates: that once is it's ON means some other similar rule is executing and we need to wait until this flag becomes OFF... 

Anyway this just a in hypothesis if and when we have such, part the rule... so it's NOT relevant now.

[Moheeb Qupty]

Hi again,

One workaround I can see, given the capabilities of Automation today, is to switch your ruleset from being trigger-based to being timing-based. The downside is that the fields you are maintaining (e.g. FixVersion) will not be updated ASAP, but merely "soon". For example:

• Rule 1 triggers every 15 minutes to scan for Stories with changed FixVersions since the rule last ran. It updates the FixVersions in Epics as needed, then quits.

• Rule 2 triggers 5 minutes after Rule 1, and does its thing (finding Epics changed since it last ran, and updating child Stories).

• Rule 3 triggers 5 minutes after Rule 2, and does its job.

Regarding your suggestion it won't work in my case, because my logic is that FixVersions can be changed only from a Story (1-1) and then need to propagate to its relevant Epics (and from Epics to Epic Parents).

Epic's FixVersions can hold all relevant Stories FixVersion (so if Epic has 5 Stories it can hold up to 5 versions in FixVersions) - means once Story's FixVersion is changed/replaced/deleted this need to be reflected in the related Epics - so my solution is to use rules triggered from rules and since I can't Branch out of a Branch. So basically I am constructing the Epics' FixVersion list again with the new change (so I had to use a Story variable or custom field to hold the Epic key... etc.

Actually all this started because I couldn't find a solution to simply manipulate a list. FixVersions is a list I need to "find" the old version of the Story, it's a bit complicated since FixVersions hold distinct versions, means if I am replacing/deleting version from Story I am deleting/replacing only if the old version isn't used by other Stories.

For example with Python I can just search the "old" version in other related Stories if exists just adding the new version to the Epic, if it doesn't exist anywhere else I need to replace the first in the list with the new version.

[Mykenna Cepek]

Again, using the Jira REST API gives you the most flexibility, as you can code things however you wish (within the confines of the API functionality).

If you wanted to keep it all within Jira Automation, then the "Advanced (JSON) Editing" capabilities of some Action Components might be worth exploring. There you can use list-oriented functions to manipulate, for example, multiple FixVersion values.

• Advanced field editing - JSON

• Smart values - JSON functions

• Use smart values to access listed values

[Moheeb Qupty]

Option 1:

Right, I wanted to explore that it gives me the ability to "remove" the old Fix Version from Epic's list and then later "add" the new version to list.

But before doing that I need to check if the old Fix ({{fieldChange}}) Version is used/exist by other Stories related to the Epic... so using if/else block w/ JQL as condition to search if other stories contain this old version is helpful. 

Option 2:

What I explained earlier and using queue might solve our synchronization issue.

[Stephen Lee]

Also, if there's a better way to solve my overall goal, I'm all ears!