How does your company handle testing & versioning Jira Automation scripts?

[Erin Quick-Laughlin]

Currently, we build scripts local to a test project, and if it's a new version, simply append a "v2" to the name.  When it passes testing, we release it to the wild, either for others to copy or to make global, plus disable the "v1".   Eventually we remember to go and cleanup the v1s.  

How does your company handle testing & versioning?

[Bill Sheboy]

Hi @Erin Quick-Laughlin 

Change management is definitely an area to improve automation rules, and here are a couple of backlog items along those lines.  They are in the Code Barrel backlog, so unfortunately we cannot vote for them.

• Support for rule configuration versions https://codebarrel.atlassian.net/browse/AUT-471
• Include full details about what changed on a rule in 'Configuration changed' audit log items https://codebarrel.atlassian.net/browse/AUT-1347

What we do is:

• Build and test rules in test projects, for both company-managed and team-managed, as needed
• To help with testing, use Jira's CSV import capabilities to load up testing issues in known testing start conditions, and use cues in the issues to guide testing
• Peer review of rules
• At this time, rules can be a bit convoluted just to solve a use case...and there are no "comment" features yet.  So we update the description and add writes to the audit log for explanations of complex rules.
• Once validated, copy rules to the target project and update for users, project JQL, etc.  We leave the original rules in the test project in a disabled state.
• Share and document rules, so the site admins and project admins are aware of them
• Periodically export and archive the full set of rules
• Actively monitor for breaches of usage from rule runaway execution

Best regards,
Bill

[Erin Quick-Laughlin]

Totally merging many points here into our documentation @Bill Sheboy !  Two questions:

1. What's an example of a cue you use to guide testing?  An updated field?  A generated comment? Other?
2. Do you append a version in the rule title? (or otherwise, how do you leave copies in the test project?)

[Bill Sheboy]

Hi Erin,

For testing cues, we put information in the imported issues' summary and description.  For example when we made rules to measure process times and Age of WIP, the summary helped the tester know which status values to pull an issue into (and the order) from the board.  The description contained what we expected to happen.  This may be overkill for many rules, and we found it helpful for complicated, chained rule execution.

We sometimes add a version to the rule title, but often when a rule has changed substantially the version is less relevant than changing the "verb noun phrase" of the title itself to reflect the improvement.  We do *always* note side-effects and known defects in the rule descriptions.

And for possible other improvements, I forgot about these two for adding REST API support for rules.  If we had this, we could probably use an automation rule to archive rule changes.  :^)

• https://jira.atlassian.com/browse/JIRAAUTOSERVER-229
• https://codebarrel.atlassian.net/browse/AUT-999

__Bill

[Liam Green]

Hi @Erin Quick-Laughlin 

I always do a test in the test project before I release it into the wild... gives me peace of mind, and hopefully stops me making silly mistakes.

I then copy it into the relevant project and make the changes needed

[Curt Holley]

Similar to all of the above....I have a test project which is where all rules are built.

They are tested and when ready copied or simply moved and configured for their "in the wild" usage.

Notes of constraints or side'effects noted in the description.

Pending on the complexity, changes to rules are either done on the fly (if comfortable with the risk) or copied back into "the lab" and tested before re-releasing.

Both of these would be great to have though.

• Support for rule configuration versions https://codebarrel.atlassian.net/browse/AUT-471
• Include full details about what changed on a rule in 'Configuration changed' audit log items https://codebarrel.atlassian.net/browse/AUT-1347

Thanks for sharing @Bill Sheboy 

[Erin Quick-Laughlin]

"Pending on the complexity, changes to rules are either done on the fly (if comfortable with the risk) or copied back into "the lab" and tested before re-releasing."

This is where we're currently at @Curt Holley - we don't put everything in the test project if we're comfortable with the risk.  

I've also considered exporting rules to store in Git, in case we need to roll back to something, rather than keeping old copies in the development project.  Not sure if it's worth it yet.

[Curt Holley]

Unless they are particularly complex rules, then exporting to Git seems a little extreme for a rule not even currently in use.

And yes! definitely rules get modified all the time "in the wild" and I don't see that as an issue.

That said, I find it easier to train newbie's up in writing and testing automation in a "lab", as they feel much safer. plus, that way I know they can't make a mess "in the wild".

Any complex rules I do tend to keep the "last known good" version of, in the Lab...for future reference as well as, if it gets messed with "In the wild".

[John Murphy]

I have a requirement to version control our automations, but I've hit a wall trying to hit the api which is used to list and export rules. 

https://<org>.atlassian.net/gateway/api/automation/internal-api/jira/<guid>/pro/rest/<projectid>/rule/<ruleid>/export

Adding a +1 for a feature to automatically export rules to a git repo. 

As for testing, we have a clone of our project for testing purposes. It creates a few small problems with variables. This could be solved with something like environment/rule variables.

[Arthur Vernon]

hi @John Murphy I have just done the above request with basic authentication - username: your email address and password being a token generated from user profile in Jira (see https://developer.atlassian.com/cloud/jira/platform/basic-auth-for-rest-apis/#supply-basic-auth-headers)

Like you I am needing a feature to backup changes to GitHub and will have to build one in lieu of an OOTB feature.