X-Ray usage feedback

Daniel Cottrell April 8, 2020

I work for a large company on various projects doing software testing. We recently started using X-Ray for all of our projects. After working with this software I've noticed several inneficiencies which could do with some improvement. We have to use it all day long so if there is something which is a bit clunky then it really slows us down.

The difficulties I've had with using X-Ray are:

1. CTR+V does not guess or determine which step to attach the screenshot to. This means you have to do ctrl+v, then select the step, then add a title, then click save.
In HP ALM we could just click the paste button, then enter our title/description for the screenshot. and click ok. I think X-Ray would be a lot better if it had a "current step".

2. After uploading a screenshot the entire page refreshes. You then have to wait for it to load fully and then manually scroll back to the step you were on, this is very time-consuming. I know that this could just be a browser restrictions. Maybe it could at least jump back to where you were previously?

3. No dedicted "Actual Outcomes" box. This is a big one. You have to Click Comment, then click the textarea box, then type your outcomes, then click the tick.
In HP Alm, we would just type what we wanted into the box and then move on. The Actual Outcomes column should be next to the Expected Outcomes one. We upload screenshots and do Actual Outcomes for EVERY step.

4. It would be good if ALM-shortcuts such as CTRL+P and CTRL+F could be used for Pass/Fail for that step.

5. The status colours are too similar for NA and Pass. This is only minor because after a while you remember that the left one is pass, but initially I got them mixed up as they were both green.

6. There don't seem to be any Run-IDs to refer to. I can see Test ID, Test Execution ID, Bug ID but nothing for runs.

7. Couldn't we use similar terminology to other testing programs? A test execution is a test instance in ALM, the term Bug is usually a Defect ("Bug" is sometimes frowned upon).

8. Please could you study ALM's features in general for use in X-Ray? ALM is buggy and awful (irony) but in some ways quite well thought through.

Some things I like in X-Ray:
1. Having overall screenshots, defects and notes.
2. Being able to navigate back to the test set (execution) by clicking "Return to test Execution"
3. Not being locked into the run and being able to navigate out to other pages and then come back again
4. Having a data column for test data
5. Having documentation and test execution in the same place.

I can't discuss any project details for confidentiality reasons. Thanks for reading.

3 comments

Comment

Log in or Sign up to comment
Bruno Conde
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
April 21, 2020

Hi @Daniel Cottrell

Thank you for your feedback.

 

Points 1 and 4 are related to having a "current step" while executing a Test. We can consider the current step the first step with a non-final status (e.g. TODO, Executing ...).

Therefore we have created the following suggestions in our backlog: 

https://jira.xpand-it.com/browse/XRAY-5470

https://jira.xpand-it.com/browse/XRAY-5471

 

I can't reproduce the behavior you are describing on point 2. When I add attachments or screenshots, the page is not refreshed and the scroll is not moving also. I suggest updating Xray to the latest version and, if the problem persists, please contact our support desk: https://jira.xpand-it.com/servicedesk/customer/portal/2

 

Regarding n.º 3, we have already released the Actual Results field in our latest version: https://confluence.xpand-it.com/display/public/XRAY/Xray+3.6.0+Release+Notes#Xray3.6.0ReleaseNotes-New%22ActualResults%22FieldonTestSteps

 

"NA" is not a native Xray status. Hence, you can just ask your admin to change the color of that status on the Xray administration page.

 

Regarding the Test Run ID, I also think this can be useful: https://jira.xpand-it.com/browse/XRAY-5472

 

I agree that the Terminology is different among Test Management tools, but this is something we can not change right now. Also, Defect is commonly used in most TM tools. Because you already have "Bug" in Jira, a Defect in Xray can be mapped to other issue types as well that are defects, not just Bug.

 

We are not trying to port ALM into Jira. We are creating a tool based on customer feedback and what we believe provides a better test management experience inside Jira.

By the way, please check out our Roadmap also: https://confluence.xpand-it.com/display/public/XRAY/Roadmap

 

Also, don't forget to vote on these suggestions and others on our backlog.

 

We really appreciate all your feedback.

Kind Regards,

Bruno Conde

Like José Domingues [Xray] likes this
Daniel Cottrell May 22, 2020

I'm really sorry I haven't responded to this previously.

We've now got an updated version of JIRA and it has fixed issue 2 and given us a new expected outcomes box which will only require 1 click to use which we like a lot.

This solves our two main issues. Thanks for adding these issues to the backlog that really means a lot to us. Will sign up if possible to register interest on these items.

Daniel Cottrell December 17, 2020

@Bruno Condeplease could you update this and increase the priority? It's only on Trivial at the moment:

https://jira.xpand-it.com/browse/XRAY-5472

Daniel Cottrell January 6, 2021

BUMP

TAGS
AUG Leaders

Atlassian Community Events