Navigating Test Execution Workflows: A Beginner's Guide to Jira Automation Rules

Heather Ronnebeck

Hello fellow Community Members! 👋 Today, let's embark on a journey into the world of Jira Automation rules, exploring a real-world use case that could save you precious time. Automation is all about efficiency, and the scenario I encountered in my environment perfectly aligns with that goal.

As a fellow customer, I want to share the challenges and discoveries from this mini-project to help you navigate similar paths without the same hurdles.

The Use Case: Automating Test Execution Creation in Xray

Use Case: Automatically Creating Test Executions with Tests assigned via Xray, based on a Story ticket that already had tests linked to it using Issue Linking.

Products Used:

• Jira Software

• Xray Plugin

• Jira Automation

Additional Items Used:

• Xray API Documentation

• Jira API Documentation

• Jira Documentation

End Result: Two automation rules to create and link Test Execution tickets within Jira and Xray seamlessly.

The Journey Begins! 

Before we dive into the specifics, let me clarify that this exploration was born out of a necessity in my own environment. Learning from this experience, I aim to share insights that can be applied broadly, potentially aiding you in tasks like attaching tests to test plans or tests to test sets within Xray.

Rule 1: Test Execution Ticket Creation

The Setup:

1. Trigger: Manual creation of a "Story" issue.

2. Conditions: Ensuring the initiator matches specific criteria.

3. Actions:

   • Create Issue: Generates a test execution ticket mirroring details from the Story issue.

   • Link Related Issue: Establishes a connection between the new test execution and the original Story issue.

   • Transition Issue: Moves the test execution to the "SIT Test Execution Created" status.

   • Refresh Issue: Updates the issue after these actions.

Rule 2: Adding Tests to Test Execution Tickets in Xray

The Setup Continues:

1. Trigger: Triggered on the Issue created event.

2. Conditions: Checks for project, issue type, and the reporter being Automation for Jira

3. Actions:

   • Refetching Issue Data 7 times

     • I did this to allow enough time for the first automation to finish linking the issues to the newly created Test Execution ticket.

   • Performing a Lookup Issues Action - issueTests = {{triggerissue.key}}

     • I needed to retrieve all the linked Tests attached to the Test Execution ticket that was created. This information would determine which tickets the web requests submitted later would be applied to and used for advanced branching.

   • Creating a Variable  - {{testid}} = {{triggerissue.issueLinks.inwardIssue.id}}

     • During testing, I realized that I needed this value in order to perform advanced branching. Therefore, I created a smart value variable for it.

     • Next time, however, I will test skipping this action entirely because of the next action in the automation workflow: advanced branching.

   • Implementing Advanced Branching

     • Here, I learned that if you have multiple items and want an automation rule to run through the same set of steps for each item, you need to use Advanced Branching.

     • The specific type of advanced branching I used was "Create Variable" because it utilized keys from my Lookup Issues Action. This allowed me to execute all steps within the branch. Once again, like my last step when creating {{testid}} variable, this rule required me to repeat that process as well since it was necessary at the end.
       This step is why I believe my previous step might be redundant. In future tests, I will explore omitting that particular step altogether.

4. Advanced Branch Steps/Actions:

   1. Send Web Request #1 – Authentication Request

      • Before we proceed with the GraphQL API, let's start by authenticating with the Xray Authentication API to obtain our token. This is a crucial step that is outlined in the Xray documentation. You can find more information about this process here. To successfully complete this automation, I followed their instructions mentioned here to acquire the necessary client_id and client_secret values.

        

   2. Create Variable - {{token}} = {{webResponse.body}}

      • This particular step is mentioned in the Xray Documentation, which I have been following since the previous step. You can also find it in the image above and the next step on the left side of the automation.

   3. Create Variable - {{testExecution}} = {{triggerissue.id}}

      • This was quite an interesting experience. I encountered challenges and learned that a new action needed to be created because the next web request action was not accepting the smart value if it contained a period.

      • I came across this issue while retrieving the id value using Jira API calls https://<sitename>.atlassian.net/rest/api/2/issue/<issue key>?expand=names,renderedFields and manually inserting that id value into the second web request action.

      • After confirming that changing the id worked, I developed this action to explore ways of automatically inserting the values. During testing, I also added an action to log this value for comparison with what was sent to the next web request action.

   4. Create Variable - {{testid}} = {{triggerissue.issueLinks.inwardIssue.id}}

      • Yes.... I understand. I am recreating this one again. I did this to ensure that this variable would work perfectly, and because I was still testing at this stage, I wanted to avoid any other issues.

      • I also discovered that variables configured in one branch do not necessarily apply to another branch, so I created a safety net for myself. By creating it here, just before my web request action, I knew it would be fresh and reliable.

      • Furthermore, during testing, I added a temporary action to log the result of this step. This allowed me to verify its functionality against the web request step.

   5. Web Request Action – Finally!

      • This web request is posting against the GraphQL API and is specifically modifying this API directly. This one is the one we’ve been building towards across this entire automation.

        

      • To make this work, we need the {{token}}, {{testExecution}}, and {{testId}} variables that were created previously. As shown above, they need to be entered into the step in different locations.

      • We also needed the actual API Code - addTestsToTestExecution which updates the test execution within Xray itself. It’s very important to take the code from the GraphQL documents and run it through a converter like GraphQL to JSON Body Converter because Jira’s automation tool doesn’t like it in the native format. I spent a bit of time learning that tidbit of knowledge too.

      • I also used the converter to add in my values both hardcoded and variables before converting it to ensure they converted properly.

      • It’s also important to delay any further execution of the rule because you need to make sure you get a response for each one of these, and yes they get logged into the audit log. I used this too for comparing with the other log actions during testing.

   6. Re-fetch Issue Data Action

      • Just to give the issue page a quick refresh so users can see that this actually worked..

Overcoming GraphQL's Limitations: Integrating Jira's Smart Values for Effective Ticket Linkage in Xray

The distinctive nature of this rule stems from its reliance on variables, offering a crucial solution to overcome GraphQL's limitations in handling smart values. Despite GraphQL's robust capabilities, incorporating Jira's smart values directly into its web requests proves unfeasible. The necessity for smart values arises from the fundamental requirement to link multiple tickets within Xray effectively.

However, a significant challenge surfaced during this process. Xray's GraphQL API, which plays a pivotal role in web requests, does not inherently support Jira's smart values. As a workaround, the smart values employed by Jira had to be transformed into variables that could seamlessly integrate into the JSON-based web requests sent to Xray. This transformation ensures that the dynamic information encapsulated in smart values remains intact and functional within the GraphQL-driven communication, enabling the successful linkage of multiple tickets within the Xray plugin.

Wrap-Up

In summary, the exploration of Jira Automation rules for Test Execution workflows reveals valuable insights and solutions. The real-world use case involving Xray integration underscores the potential of automation to enhance efficiency.

The two rules crafted exemplify a systematic approach to streamline Test Execution ticket creation and linkage within Jira and Xray. Rule 1 establishes the foundation, mirroring details from a manually created "Story" issue. Rule 2 focuses on handling multiple items efficiently, incorporating Advanced Branching and variables for a comprehensive automation workflow.

Challenges, notably in overcoming GraphQL limitations and utilizing variables for smart values, highlight the need for adaptability in testing workflows. The guide emphasizes the importance of leveraging tools like variables to enhance automation rule capabilities.

In essence, this beginner's guide offers practical insights for those entering Jira Automation for Test Execution workflows, empowering them to navigate similar paths more efficiently. Here's to streamlined automation in your testing endeavours! 👏

Curt Holley

Love it. Though I'm not sure this is where I would get a beginner to start with automation 🤯