Can We Import Tests From A Test Set Into A Test Execution Using Jira Automation?

[Ste Withington]

Hi. Has anyone discovered how to use Jira Automation to import test cases stored within a Test Set into a Test Execution?

These ticket types are from Xray.

[Theresa McFarlane]

Unfortunately, Jira Automation doesn't have a built-in function to directly import test cases from a Test Set into a Test Execution for Xray. However, there are a couple of workarounds you can explore:

1. Utilize the Jira REST API:

You can write a script (e.g., Python with JIRA library) that interacts with the Jira REST API.
The script can retrieve the test cases within a Test Set and then create them within a Test Execution using the relevant API endpoints.
This approach requires some coding knowledge but offers more flexibility.
Resources to get started:
Xray REST API documentation: https://docs.getxray.app/display/XRAY/REST+API
JIRA Python library: https://jira.readthedocs.io/
2. Leverage third-party add-ons:

Consider using add-ons like Zephyr Squad or Xray Lite for Jira. These add-ons might offer features to import test cases from Test Sets to Test Executions.
Explore their functionalities and documentation to see if they align with your needs.

[Stefan Salzl]

Hi @Ste Withington - welcome to the community

let´s break down some terminology:

• test case is a step by step description of a repeatable procedure to do something in order to test a special feature/behaviour
• a test set is just kind of a "physical" container that collects several test cases
• a test execution is a container of really running/executing a collection of or even one test case(s)
  • create an execution out of a single test case
  • create an execution out of a test set

So regarding the last point in my list creating a test execution from a test set is a main functionality in xray. It creates executable "test runs" of your test cases which are grouped together in a test execution issue.

Therefore the question is:
What would be your exact use case? would you like to create test runs of your test cases into an existing test execution issue? Or should a new test execution issue created out of the test cases of a test set? 

You mentioned it should be done automatically:
When should this be done? What is the trigger for that?

Best
Stefan

[Ste Withington]

@Stefan Salzl I'm a long time QA Manager (~12 years) and an even longer time Jira user (~15 years) having used Xray to manage my teams test effort for the last ~4 years, so I'm fully aware of the terminology and how we use the tickets. But thank you for confirming.

To be clear. my question is regarding the 'Jira Automation' tool. I already have over 20 Jira automations created and running daily, but I have one specific automation that I cannot work out.

Let's break it down for you;

- I already have a test set which contains many test cases.
- Using Jira Automation, I would like to import those test cases into a new test execution ticket.

I can do this manually via the 'Add Tests > Tests from test sets' function, which works great. But I want to automate it using Jira Automation.

Thanks

[Stefan Salzl]

Hi @Ste Withington 

impressive 😉🙌 we kinda have the same background

Out of the box A4J is can´t handle xray operations natively. It can be done with a "send web request" action in the atomation and utilising the graphql API which is pretty powerful in cloud --> see the link. 

https://us.xray.cloud.getxray.app/doc/graphql/gettestset.doc.html

with the above mentioned method you could eg. grab all tests (more concret issueIds of tests) in a specified testSet.

I already tested the following call (which gives the testIds of the last created testSet):

{"query": "query{ getTestSet{ issueId projectId tests(limit:100){results{issueId}}} }"}

to get only the list of test issueIds you could fetch the webResponse and filter through the nodes with the following statement:

{{webhookResponse.body.data.getTestSet.tests.results.issueId}}

this list could then be saved in a variable and then again sent via another webrequest and agian with a graphql call to add those tests to a specified testExecution issue.

Should the testExecution be created in the automation rule too? if not: how could the automation rule know which testExecution to choose? what would be the trigger for the automation rule? 

Best
Stefan

[Ste Withington]

@Stefan Salzl thanks for your time and the info on retrieving the tests from a test set using a call. I will find some time to investigate.

In response to your questions:

Should the testExecution be created in the automation rule too? if not: how could the automation rule know which testExecution to choose? what would be the trigger for the automation rule?

I already automatically create a testExecution when a new version is created. From that I would be happy to either extend the rule to perform the call you mentioned or equally I'd be happy to trigger the rule manually.

[Stefan Salzl]

Hey @Ste Withington 

thanks for the feedback. I would be happy to see this working on  our side.

Just give it a shot an let us know if you got this working or have any further questions.

Best
Stefan

[Vasileios Theodorou]

I used this query to get the Tests from a Test set:
{"query": "query{ getTestSet(issueId: \"{{testSetId}}\") { tests(limit:100){results{issueId}}} }"}

Saved the results in the variable testsFromTestSet:

{{webhookResponse.body.data.getTestSet.tests.results.issueId}}

And then added the Tests to the Test Execution by using this query (notice the string manipulation required to pass the list of Test Ids as returned by the previous query):
{"query": "mutation { addTestsToTestExecution(issueId: \"{{testExecId}}\", testIssueIds:[\"{{testsFromTestSet.replace(" ", "").split(",").join("\\\", \\\"")}}\"]) { addedTests warning } }"}

It worked :)