How to create xray - test execution report on issues in Jira cloud

Vikas S April 6, 2022

Hi, 

I have used below query to get entire test(Issues) from test execution, then i would get result and count.

Filter Name 1  : Project= ABCD AND issuetype= Test Execution AND label: abcd

Filter Name 2 : Project = ABCD AND Issue in testExecutiontests("Filter Name 1")

 

 

But when I go to X-ray - report options and select test execution report and used Filter Name 2 inside saved filter drop down and used generate report option. then i got No records found message and When i run Filter Name 1 in same Xray -report window and select test Execution report and click on generate report button then data is showing for first filter.

 why data is not showing for Filter Name 2  that is second filter.

Please help me on this. Thanks

2 answers

2 votes
Mohajit Acharya April 6, 2022

@Vikas S , are you using the right syntax for testExecutionTests function? Please check the below link. Another parameter is required when you use a saved filter.

https://docs.getxray.app/pages/viewpage.action?pageId=62267611

Also, looks like testExecutionTests function is not yet implemented for Jira Cloud. (I am assuming you are using Jira Cloud and not Server). See the below confirmation from Xray Support Team.

https://community.atlassian.com/t5/Marketplace-Apps-Integrations/How-to-find-XRay-Tests-in-Test-Execution-Jira-Cloud/qaq-p/1259204

As an alternative, you may want to use the Test Runs List report:  

https://docs.getxray.app/display/XRAYCLOUD/Test+Runs+List+Report

It provides additional detail since it will list at the Test Run level; however, all the Tests will be there. It is possible to filter by several fields, including Test Execution keys, using saved filters.

Moreover, Xray has a GraphQL API that allows retrieving most of Xray's information. Please check the documentation: https://docs.getxray.app/display/XRAYCLOUD/GraphQL+API

In case you want to get several Test Executions at once, based on JQL, there is also this query getTestExecutions, and here is the respective documentation: https://xray.cloud.xpand-it.com/doc/graphql/gettestexecutions.doc.html

Vikas S April 6, 2022

Hi @Mohajit Acharya 

what I need is not test execution list inside my test execution report but I want to create a report which has list of tests(issues) inside my test execution report, why I need it is for test runs has same tests are repeated twice or thrice, but in List of Tests when we run filter no 2 that has non repeated test cases it is grouped by default. So it resembles the overall test results gadgets data but it should not same as test runs summary gadget data.

hope you understood. Let me know how to crack it. Thank

 

Regards,

Vikas

0 votes
Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 6, 2022

Hi @Vikas S ,

Could you define your requirement in more details and from a use case perspective? What would you like to accomplish? That would make it easier to understand what you tried already and finding a helpful and fitting solution.

Best
Stefan

Vikas S April 6, 2022

Hi @Stefan Salzl 

I need to generate test execution report by using xray- report option in jira cloud.

First of all we have two filters, 1st filter consolidate list test executions for particular project for ex- : Project= ABCD AND issuetype= Test Execution AND label: abcd,

project ABCD is name of the project and in that project we have used a filter to consolidate test execution based on Label. And it shows around 17 test executions, and with respect to this test executions we have 165 test cases.

 

In Filter no 2 : Project = ABCD AND Issue in testExecutiontests("Filter Name 1")

we create Filter no 2 under same project which exports all tests(test cases) under filter no 1 condition (test executions).

 

So i would like to generate test execution report using Filter no 2 and the result count should match with my overall test result gadgets which is already create or built in my dashboard.

But if i used Filter no 2 it is showing No records found. I am just confused. So please find the attached snap for referenceScreenshot (78).png

Craig Nodwell
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 6, 2022

Just throwing this out there but doesn't Filter Name 1 already satisfy the Project = parameter?  Have you tried trimming Filter Name 2 to exclude the "Project = abcd AND"

Vikas S April 6, 2022

Yes, I tried but not working.

Ok let me know exporting test cases in test execution report is that possible.. Because I am trying this out for a first time.

 

(Project = ABCD AND Issue in testExecutiontests("Filter Name 1")  -----> In this filter I am getting No record found in xray -report >  test execution report> Generate report.

but in Jql search box i got results for filter no 2.

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 6, 2022

Could be that I get something wrong here:

The 1. filter has a list of test executions as output. So from my point of view this would be relevant for a test execution report as this would provide metrics/information regarding test execution (issues):

https://docs.getxray.app/plugins/servlet/mobile?contentId=62267415#content/view/62267415

 

The 2. filter outputs a list of test cases. So I‘m not sure how a test execution could provide metrics when the base (input) are test cases. In this case maybe the test run list is what you are looking for. This will provide information which testcase was running or is planed to run in certain test executions:

https://docs.getxray.app/plugins/servlet/mobile?contentId=37062067#content/view/37062067

 

Hope I could give some valuable input.

Best
Stefan

Vikas S April 6, 2022

Thanks @Stefan Salzl

what I need is not test execution list inside my test execution report but I want to create a report which has list of tests(issues) inside my test execution report, why I need it is for test runs has same tests are repeated twice or thrice, but in List of Tests when we run filter no 2 that has non repeated test cases. So it resembles the overall test results gadgets data but it should not same as test runs summary gadget data.

hope you understood. Let me know how to crack it. Thank

 

Regards,

Vikas

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 7, 2022

Sorry but out of the description I don´t have any idea what you are trying to accomplish (or why) =S

Could you eg. provide a screenshot where you are trying to resolve that and what you would expect to see?

Vikas S April 7, 2022

@Stefan Salzl 

Ok. let me know that, can we export testexecution report for test case (test) list after execution of filter no 2.

I just wanted to export test case in test execution report by using Xray- report option available in left side menu bar. And my project is created under Kanban Framework.Screenshot (83).pngScreenshot (87).png

In the First snap - I am getting all test execution in ABCD project as per label mentioned.

In the Second snap - I am getting all tests of filter 1.

 

I just wanted to run my second filter inside test execution report and i did but results shows No record found.

 

 

I hope you are getting the scenario now..

 

Thanks

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 7, 2022

I got that. As described a test execution report can only take a list/filter of test executions to analyze/report them. Your approach is feeding the functionality with wrong input. 

Just from what I guess you are trying to achive:

If you´d like to feed a list of tests (issues) and want to get back a report that lists metrics of these tests and where they were executed (1 entry per test run) then I would suggest to use the test run list. Attached a screenshot of an example I did:

  • filtered all tests for a specific story
  • filter:

image.png

  • report

image.png

 

Best
Stefan

Vikas S April 7, 2022

Ok, I am new to jira anyway, I need test distribution status as in below screenshot that is Total test case, Pass, Todo, Executing, Fail, Aborted. So how to get generate report for test list on this status see below attached image .TestExecutionsReport_PrintScreen.PNG

I need it for all test cases(Tests) and it should match with overall test results gadget data in my dashboard. But in Test runs report i doesn't found any status like Todo, Pass, Executing, Fail etc... But in test execution report i saw them. But I tried in test execution inside test execution report but count I mean testcases are repeated under different test execution it depends on requirement but it shows more count because we have lots of repeated test cases.

Thanks

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 7, 2022

The list in your screenshot is showing a list of test executions (not test issues).

I´m really sorry but there is too many different terms mixed up and I´m afraid I´m not able to follow the description. Please check documentation of either test run list and test execution report (both for cloud) and see what they are capable of.

In case you are convinced that your requirement should be covered according to the documentation please contact vendor support for further details:

 https://jira.getxray.app/servicedesk/customer/portal/2/user/login?destination=portal%2F2

 

Best
Stefan

Vikas S April 7, 2022

Yes correct, And i want to it for test issues(tests) i couln't run filter no 2 inside this test execution report and it shows No records found any other ways to get only test cases which should not repeated, I want it in group by test cases should display only one time inside test execution report for exact count matches Overall test Results gadget data. But i herd group by is not working in Jira

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 7, 2022

Currently this seems not possible out of the box. The test execution report is only capable to show a list of test excution issues with according metrics for the test execution issue itself.

Vikas S April 7, 2022

Anyway thanks! for your time spend here for me. I learned atleast something.

Like Stefan Salzl likes this
Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 7, 2022

You are welcome 🙌🏼

Stefan Salzl
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 19, 2022

@Vikas S 

Please consider to click the accept button in order to mark this issue as solved and make it easier for other users to find a solution in case they are facing similar problems.

Best
Stefan

Suggest an answer

Log in or Sign up to answer