Hacking/Parsing Automation Rules with JSON for Fun and Profit

giphy

Automation Rules can be exported and imported as JSON files, which is useful if you wanted to copy/share rules across Jira instances. But this feature can be exploited for some other tasks probably not intended by the Code Barrel guys.

Some examples, which I've detailed below:

  • Finding all rules where a user is being set to Assignee or another user field. Very helpful when someone leaves the company. (Thankfully Atlassian saw fit to add a Transfer User option, which lets you replace all references to one user with another. If you need to perhaps do this selectively, then my tips on that might still be helpful.)
  • Validating repeated elements in rules, like link types.
  • Cataloging your rules in Confluence.

Some other ideas, not explored here:

  • Using tools like diff or even Bitbucket to track changes in rules.
  • Bulk-creating rules based on templates.

Note - this was primarily tested on Automation for Cloud, but Automation for Server/DC also allows export and import of rules.

Tools that you will need:

  • jq - an incredibly powerful command-line tool that can read JSON files. It can be tricky to learn and use, but is incredibly powerful.

Optional

  • JSON Viewer (Chrome extension for viewing JSON files)
    • You'll need to enable it to "Allow access to file URLs" to view downloaded rules.
  • Firefox's built in JSON viewer

Export Automation rules from Jira

So you can export all of the rules from the global configuration page for Automation (great for backups), but you can also export an individual rule:

Screen Shot 2022-03-26 at 1.46.42 PM.png

Parse Automation Rules

Either way, when you download the exported rule(s) from Jira, they will not have any linebreaks, so it will be very hard to read or edit them. Firefox has a nice JSON parser/viewer built-in, so if you just wanted to read the rule, you could drag it there, or into Chrome if you have the JSON Viewer extension installed.

To parse the file into something editable, you can use a very basic jq filter to reformat the JSON and then write that into a file for editing:

jq . automation-rule-1128847-202009232238.json > nicelyformatted.json

So, within that JSON you can see things like:

"name": "CI -> IN PROGRESS - Create Request (SDES): SAP",
"state": "ENABLED",
"description": "Create SAP Task (ATLAS-1621, ATLAS-1481)",
"authorAccountId": "557058:5...",

You can look up that AccountId by appending it to the end of your directory URL, like this https://MYSITE.atlassian.net/jira/people/557058:5..., you’ll see that’s me:

Screen Shot 2022-03-26 at 1.46.50 PM.png

You will also see Ids like this for assignee, which is useful below:

Look for an user in all rules

Instead of all the work below, it's probably better to look at the Transfer User feature that lets you do a global replace of a user who has left. It includes a Preview function that will let you see all the rules where a user is currently referenced, and even if you have to reassign rules/user fields to different people, if it's not too many rules, you could probably do it manually.

So, if you can export all the rules, and you can parse those rules, then you can find every rule where a user is set to Assignee. Handy when somebody leaves and you’re not sure what rules they are included in.

There is probably a very elegant way to use jq to walk through the JSON structure and search for assignees, but I’m not very elegant, so I did this:

jq . automation-rules-202008312215.json |egrep '^  "name":|ACCOUNTIDFORKELLY' |less

What that does is print the Name of every Rule, and if it finds the ACCOUNTIDFORKELLY, it prints that too. So you end up with output listing all the Rules, and some of them have that Kelly's account ID, meaning they are referenced somewhere in that rule.

Here’s an excerpt:

      "name": "Auto assign on transition",
      "name": "Balance support load for team",
                  "value": "ACCOUNTIDFORKELLY"
...
      "name": "Offboarding - Sub Tickets Creation",
                  "value": "ACCOUNTIDFORKELLY"
                  "value": "ACCOUNTIDFORKELLY"
                  "value": "ACCOUNTIDFORKELLY"
                  "value": "ACCOUNTIDFORKELLY"
                  "value": "ACCOUNTIDFORKELLY"

If you open those rules up, you’ll see that Kelly is somehow in those rules, as part of a User List for balancing assignments, or as the main assignee for sub-tickets.

Bulk replace an Assignee

Again, instead of all the work above/below, it's probably better to look at the Transfer User feature that lets you do a global replace of a user who has left. It includes a Preview function that will let you see all the rules where a user is currently referenced.

Now, if somebody’s replacement was going to take over all of their ticket assignments, it’s fairly easy to do a search and replace across all the rules in the JSON export with the text editor of your choice, and then import that back into Jira.

Validate Link Types

So maybe somebody has hand-crafted 88 rules(!) each of which can create 5-6 different subtasks. But halfway through creating them, they realized they incorrectly set the link type for each of those subtasks to "Blocked by" (inward link) instead of "Blocks" (outward), and they had to go through the previous 40-some odd rules (by hand) to change the link type (in 5-6 different subtask creation blocks).

And when they were done, they wanted to validate that all link types were properly set.

Well, it would be nice if all of the rules were in the same project, so you could select for just those, but no, these are all global rules. Luckily the creator used a standard naming convention for all the rules (# - SCRXXX ... ), so we can filter that way. Let's get to it:

Ok, let's make sure we're only getting rules that match the naming convention:

jq '.rules[] | select(.name | contains(" SCR")) |.name' automation-rules-202009302350.json

Returns names for 88 rules. 

And if we remove the last .name part we will get all the rules. Let's pipe those through less:

jq '.rules[] | select(.name | contains(" SCR"))' automation-rules-202009302350.json |less

So looking inside one of those rules, we see this bit:

          "component": "ACTION",
          "parentId": "22954403",
          "conditionParentId": null,
          "schemaVersion": 9,
          "type": "jira.issue.create",
...
                "field": {
                  "type": "ID",
                  "value": "issuelinks"
                },
                "fieldType": "issuelinks",
                "type": "SET",
                "value": {
                  "issue": {
                    "type": "COPY",
                    "value": "trigger"
                  },
                  "linkType": "outward:10000"
                }

A quick check of the Issue linking confirms that "linkType": "outward:10000" is Blocks

Then it's just a bit of grep goodness:

jq '.rules[] | select(.name | contains(" SCR"))' automation-rules-202009302350.json |grep '"linkType"'|less

And there we have all the linkTypes for all of those rules, and can confirm that they are all "linkType": "outward:10000".

Importing JSON rules

The Import functionality in Automation Rules is pretty smart, in that if it detects a JSON of multiple rules, it will let you select which ones you want to import. Additionally, if there is already an existing rule by that name, it will rename the rule “Copy of…” (although in some recent testing this did not happen.)

Most importantly though, any imported rules are Disabled by default. So you should be able to review them to confirm the new assignee, etc. before enabling the new rule and disabling the one you’re replacing.

Cataloging Automation Rules

csvrules.pl

csvrules.pl takes a full export of Automation Rules and converts it to a CSV with Name, Link, Description, Enabled, Project Key, and Project Name that is suitable for parsing with the Confluence Advanced Tables add-on (hyperlinks Rule Name to the link for the rule.)

It references projects.csv to lookup Project Names by IDs.

I use Bob Swift's CLI to generate that list:

acli --action getProjectList --columns "id,key,name" > projects.csv

But you could also look them up via an API endpoint:

https://YOURSITE.atlassian.net/rest/api/3/project/search

Although if you have more than 50 projects you'll need to do the whole pagination thing. Once you get the JSON though, jq can output it as nice CSV:

jq -r '.values[] | [.id, .key, .name] | @csv' projects.json

 

9 comments

Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 26, 2022

I apologize for dumping a pretty dense and technical set of text up there, but this has been sitting around for a while, and I figured it'd be good to at least get the information published in case anybody ends up searching for things like this.

To be clear - learning jq and parsing JSON can be incredibly daunting, and I'm always having to go back to my previous work to figure out/remember exactly what the heck I did.

But if you end up writing complex Automation Rules, being able to have some way to look at them as "code" outside of the editor can be very useful, at least to avoid clicking a bunch of times to see if you set the right link type, or put the correct person as Assignee in every IF clause, etc.

Anyways though, if you have questions about how jq works, or wanted help with parsing your own rules or doing something interesting with them, I'm happy to discuss that in this thread, or you can always ask a question.

Like # people like this
Bill Sheboy
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 28, 2022

Thanks and well done, @Darryl Lee 

Still on my wish list: a refactoring IDE with version control native for automation rules edit...making such incremental changes easy and safer.  Or a full REST API for rules and then someone can create an addon (such as for VS Code) to improve rule maintenance.  One can hope...

Would you consider posting an image of how you catalog rules in Confluence?  I went "brute force" to document any convoluted logic with text/images and included the latest JSON export...just in case.  Thanks!

Kind regards,
Bill

Like Steffen Opel _Utoolity_ likes this
Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 28, 2022

Thanks @Bill Sheboy Yeah I don't know if we'll ever see a full IDE or even REST API for this, although we can dream, right?

ANYWAYS, regarding my rule's cataloging, my little Perl script outputs CSV that looks like this:

Name, Description, Enabled, Key, Project

"[Add lusers | https://YOURSITE.atlassian.net/plugins/servlet/ac/com.codebarrel.addons.automation/cb-automation-project-config?project.key=DELETED&project.id=#/rule/2681068]", "Add multiple users to a custom field, using JSON.", ENABLED, DELETED, ""

"[Add request participants | https://YOURSITE.atlassian.net/plugins/servlet/ac/com.codebarrel.addons.automation/cb-automation-project-config?project.key=HELP&project.id=10001#/rule/1819286]", "", ENABLED, HELP, "HELP"

"[Advanced comparison | https://YOURSITE.atlassian.net/plugins/servlet/ac/com.codebarrel.addons.automation/cb-automation-project-config?project.key=DELETED&project.id=#/rule/2598558]", "Send email if it estimate has exceeded 30 days.", ENABLED, DELETED, ""

Appfire's Advanced Tables for Confluence has a nifty CSV (Comma Separated Values) macro that will convert a CSV to a nicely formatted table. (I'm using wiki markup for the link, so you have to set the Output format to wiki.) and you end up with this:

Screen Shot 2022-03-28 at 1.15.53 PM.png

This is an admittedly a very high-level "catalog", and can only reflect what is in the export file. But once set up, if you add/remove any rules, you can just copy/paste the CSV output by my script and it'll be "updated".

(Yah yah Perl. Anyone out there is welcome to convert it to Python.)

If you need to add additional notes, images, etc., then it probably makes more sense to use this in conjunction with an Excel or Google Sheet which can have supplemental columns/fields for diagrams, and a link to the actual JSON (each rule split out and checked out into a source control system). :-}

Like # people like this
Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 28, 2022

And speaking of APIs, as previously mentioned, I couldn't even get this endpoint to work for exporting the darn rules:

https://MYSITE.atlassian.net/gateway/api/automation/internal-api/jira/<CLOUD-ID>/pro/rest/GLOBAL/rule/export

(Like, it works from the browser where I have an active session, but I cannot get it working with Bob Swift CLI's renderRequest, which normally lets you "scrape" other types of content.)

I found that endpoint when I select "Export rules" from the Global Automation Rule's UI:

https://MYSITE.atlassian.net/jira/settings/automation#/rule-list?systemLabelId=all&page=1&pageSize=20

When using renderRequest with both of those pages (using an API token, of course), I ended up with what looks like a bunch of SPA (Single-Page Application) code. Bleh. 

If anybody else is better versed in scraping content, I'd be much obliged to find a way to automate exporting rules. Perhaps @Bob Swift _Appfire_ might have some ideas?

Bill Sheboy
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 28, 2022

Hey, Darryl.

I would not recommend brute-force scraping this, as the rules UX appears to be evolving.  (e.g. It appears an attempt is being made to solve the rules list performance issues with pagination.  At least it doesn't hang any longer when I open my test project's rules  :^)

Where did you find documentation on that REST API method to get the rules?  Perhaps there is something described there to indicate the symptom you are observing.

Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 28, 2022

@Bill Sheboy hehe. Documentation? For something in the path internal-api? Unlikely. :-}

But since it does have /gateway/api in the path, I was hoping there was a chance that it could be obtained programmatically.

As mentioned, the URL/method was found by watching the network requests in Chrome when I dropped down the menu and chose "Export rules".

I found mention of "/gateway/api" in documentation for Atlassian's GraphQL API

 https://jdog.atlassian.net/gateway/api/graphql

Somebody else poked around and found another /gateway/api call that allowed them to export users to programmatically export all users to CSV:

https://admin.atlassian.com/gateway/api/adminhub/um/site/<CLOUD-ID>/users/export

Oh, it is very cool that now I know that the KEY in my string is actually my CLOUD-ID, visible in places like the Manage Access screen for Products. I may try modifying the Python script from that page and seeing if it works. :-}

Like Bill Sheboy likes this
Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 27, 2023

Hey hey, this post is nearly a YEAR OLD!

Well, as a pre-birthday present I took a closer look and realized (DOH) that you can programmatically export your rules. I'm just a dummy and was looking at file output instead of the response data.

So then what you want is:

acli --action renderRequest --requestType GET --url "https://YOURSITE.atlassian.net/gateway/api/automation/internal-api/jira/YOURORGANIZATIONID/pro/rest/GLOBAL/rule/export" > automationrules-20230327.json

YOURORGANIZATIONID is what you see when you're inside your Org at https://admin.atlassian.com/

One caveat is that CLI will prepend this bit of text to the output:

Rendered data for https://YOURSITE.atlassian.net/gateway/api/automation/internal-api/jira/YOURORGANIZATIONID/pro/rest/GLOBAL/rule/export

And JSON tools like jq are not really into that. If you're on a Mac or other machine that has standard Unix tools, you could pipe the output through the handy-dandy tail command which can give you all the lines starting with line two. Oh and let's also automatically add the date using the date command:

acli --action renderRequest --requestType GET --url "https://YOURSITE.atlassian.net/gateway/api/automation/internal-api/jira/YOURORGANIZATIONID/pro/rest/GLOBAL/rule/export" | tail +2 > automationrules-`date '+%Y%m%d'.json

Run that on a schedule somewhere, and voilà, you've got automatic backup of your rules! Happy birthday!

Like # people like this
Andras M_
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
May 4, 2023

Hey @Darryl Lee - I came across this article by chance but saw it had some good points to it. I saw your update from earlier this year. While I've not used Bob Swift's CLI tool, but the command line tool cURL should work to pull the JSON data down.

 

The rules themselves will be part of the rules array. The following should work and shouldn't write additional data to the JSON file:

curl --location 'https://your-site.atlassian.net/gateway/api/automation/internal-api/jira/{cloudId}/pro/rest/GLOBAL/rule/export' \
--header 'Content-Type: application/json' \
--header 'Cache-Control: no-cache' \
--header 'Authorization: Basic redacted' > automation-rules-$(date '+%Y%m%d').json

 

The same can be setup using a cross platform tool like Postman.

 

Additionally jq is a great tool especially when interacting with the Automation JSON export file. I'll share a few queries I've written up to help troubleshoot issues in the past.

 

Search for automation rules with a specific trigger:

jq '.rules[] | select(.trigger.type | startswith("jira.issue.field.changed")) | select(.trigger.value.actions[] != null)' file.json

In the above example, we're searching for automation rules triggered by Jira Issue Field changes, in this case for rules that have an issue operation (create, edit, assign, transition issues) configured. If the rule was set to listen for all changes to the field, the expectation would be the .trigger.value.actions[] array would be empty (null).

 

This next one breaks down the enabled rules based on trigger type:

jq '[.rules[] | select(.state == "ENABLED")] | group_by(.trigger.type)[] | {triggerType: .[0].trigger.type, count: length}' file.json
{
"triggerType": "cmdb.object.trigger",
"count": 2
}
{
"triggerType": "devops.deploy.event.trigger:statechange",
"count": 29
}
{
"triggerType": "jira.incoming.webhook",
"count": 3
}
{
"triggerType": "jira.issue.event.trigger:commented",
"count": 9
}
{
"triggerType": "jira.issue.event.trigger:created",
"count": 57
}
{
"triggerType": "jira.issue.event.trigger:transitioned",
"count": 36
}
{
"triggerType": "jira.issue.field.changed",
"count": 14
}
{
"triggerType": "jira.jql.scheduled",
"count": 11
}
{
"triggerType": "jira.manual.trigger.issue",
"count": 70
}

This can be helpful during automation rule audits when completing cleanups or when looking to optimize automation rules, aiming to get a holistic view of the various triggers in use and whether it could be contributing to issues with performance.

 

This one is fairly simple, it gives a total count of enabled rules based on the export file:

jq '[.rules[] | select(.state == "ENABLED")] |  length' file.json
231

 

Search for a specific rule actor using Atlassian Account ID:

jq .'rules[] | select(.actor.value == "6edd6ccaaade4a996b9422f2")' file.json

 

I hope these examples will be helpful, as well as provide some additional examples on how parsing the exported automation JSON file can be beneficial in not only troubleshooting issues but help manage and maintain rules.

Like # people like this
Darryl Lee
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 4, 2023

@Andras M_ thanks for the curl command!

I get so used to using CLI for other things and not having to mess with authentication, I totally forget that for API calls, API Tokens work perfectly fine with basic auth. (I see you included a Authorization: Basic header, but not everybody knows how (or wants to) to base64 encode their userid:token :-).

SO THEN to make it easier, after obtaining an API Token folks could just run this command:

curl --location 'https://your-site.atlassian.net/gateway/api/automation/internal-api/jira/{cloudId}/pro/rest/GLOBAL/rule/export' \
--header 'Content-Type: application/json' \
--header 'Cache-Control: no-cache' \
--user me@example.com:my-api-token > automation-rules-$(date '+%Y%m%d').json

 And ah, thank you for more examples of jq. Love that tool!

Like Andras M_ likes this

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events