Xray - Is there a way to show a summary of Test Step results at the execution level?

Question

In Xray - All of my tests have multiple test steps. When I run an execution on a test, I would like to see a breakout of the individual test step results rather than just one result for the execution as a whole (ex. Test Steps: 8 pass, 1 fail, Execution Status: Fail). is this a feature of Xray, or something that could be configured? I need the ability to report out the status of all Steps in an Execution/Plan.

Also, In addition to the default Pass/Fail/To Do/Executing, I would like to set up 2 Custom Execution Statuses (Blocked, and Deferred) to be able to assign at the Test Execution and Test Step level. Then I'd like for the overall Execution Status to trigger off of some manual rules ->

* Is it possible to configure Xray to show the overall Test Execution Status as "Blocked" when 1 or more test steps within that execution is "Blocked"? For example, 9 total test steps, 8 "Pass" but 1 "Blocked," show overall Test Execution Status = "Blocked."
* And I'd want "Deferred" to essentially have no impact on the status of the execution. For example, say I have 9 steps, and 8 "Pass" but 1 is "Deferred" - i'd like the whole Execution status to show "Pass."

Any help is appreciated. Thank you!

Rogério Paiva - Xray Xporter · Answer

Hi,

Currently, it is impossible to have a status summary of the Steps status in the Test run screen. Please contact Xray Support (http://Xraysupport.getXray.app) to share your use case. We would like to have a chance to analyze it.

However, it is possible to create a report using the Document Generator, which has a code that can help build a template to generate the result you need.

Xray Cloud: https://docs.getxray.app/display/XRAYCLOUD/Template+Building

Xray Data Center/Server: https://docs.getxray.app/display/XRAY/Template+Building

And you can also set your won status:

Xray Cloud:

https://docs.getxray.app/display/XRAYCLOUD/Global+Settings%3A+Test+Statuses

https://docs.getxray.app/display/XRAYCLOUD/Global+Settings%3A+Test+Step+Statuses

Xray Data Center/Server:

https://docs.getxray.app/display/XRAY/Manage+Test+Statuses

https://docs.getxray.app/display/XRAY/Manage+Test+Step+Statuses

And yes, it is possible to create a Step Status that does not affect the Test Execution Status. When creating a new Test Step Status, set the respective Test Run Statu as PASS; this way, it will not affect the overall result because PASS is the status with the lowest influence in the result. Any other status will have a higher weight over the status PASS.

Thank you.

Kind regards,
Rogerio Paiva [Xray Support Team]

SivaKartheek · Answer

I am also looking for same

Nick Olsen · Answer

We are looking to add custom execution statuses as well. Were you able to do this?