Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,299,120
Community Members
 
Community Events
165
Community Groups

Comparison between Zephyr & Xray

I am sharing this article with both Zephyr and Xray information and everything should be in one place. It will be easy to know when selecting Zephyr or Xray.

Zephyr and Xray are both Jira plug-ins used for test case management i.e., test case writing and execution purposes. Both do almost similar kinds of work with some advantages and disadvantages of them with respect to cost, automation, reporting, etc. 

Here is the feature level comparison for Zephyr and Xray

1.PNG2.PNG

Reports:

Zephyr provides Traceability Report, Test Case Execution Report, Test Execution Burndown Chart and Top Defects Impacting Testing, etc. whereas Xray provides Traceability Report, Overall Requirement Coverage Report, Historical Requirement Coverage Report, Test Plans Report, Test Executions Report, Test Runs Report, etc.

Cost:

JIra pricing.png

Reference Links:

Features: https://interworks.com.mk/comparison-of-jira-test-management-plugins/

Zephyr Advantage: https://www.getzephyr.com/insights/why-zephyr-jira-1-alternative-xray

Zephyr Details: https://zephyrdocs.atlassian.net/wiki/spaces/ZFJ0300/overview

Xray Advantage: https://www.getxray.app/xray-vs-zephyr/

Xray Details: https://confluence.xpand-it.com/display/public/XRAY/About+Xray

Zephyr Pricing: https://marketplace.atlassian.com/apps/1014681/zephyr-for-jira-test-management?hosting=server&tab=pricing

Xray Pricing: https://marketplace.atlassian.com/apps/1211769/xray-test-management-for-jira?hosting=server&tab=pricing

26 comments

Regarding versioning:

The approach Xray is using is not really versioning that people expect. You cannot just select that you want to include version 2 into one test execution and version 3 of the same test in another test execution. Versioning is only supported in the way Jira is doing it.... But you definitely get a frozen list of test steps in a test execution that won't be changed if you change the test case afterwards. Which is most of the time enough, also for enterprises and big projects/programs from my experience. See https://docs.getxray.app/display/XRAY/Test+Versioning

Real versioning of test cases is only possible if you don't use Jira issues to store the date like it is done in for example in QMetry Test Management for Jira (https://marketplace.atlassian.com/apps/1215144/qmetry-test-management-for-jira?hosting=cloud&tab=overview)

Like Bernhard Renner likes this
Taranjeet Singh Community Leader Dec 06, 2020

This is very useful comparison of Xray and Zepyhr apps. Thanks for sharing this @Jayesh Wankhade !

Gald that you liked it @Taranjeet Singh and helpful for you. Thanks for motivating me. 

Hi, @Jayesh Wankhade  could you send me the snapshots from 'Zephyr vs Xray" comparison?

Unfortunately, I can't correctly displays Features 1 and 2 you posted.

 email: pwilkowski@o2.pl

Like Jayesh Wankhade likes this

Thanks @Piotr piotr for pointing it out, and sorry for the late response. I have updated the images here as well and sent you on given email. Please let me know if you received it not.

Like Piotr likes this

Thanks @Jayesh Wankhade for sharing your tool comparison sheet!

Like Jayesh Wankhade likes this

@Jayesh Wankhade @Piotr This is outdated feature comparison list. ZSquad (brand name got updated as well) offers all the test management, automation related features.

Attaching the link for same: Feature-Link 

Like # people like this

Thanks, @Harsh Dubey for highlighting it. This blog was written a year back. 

Like Harsh Dubey likes this

Hi @Jayesh Wankhade

Would you mind updating the table? I think this is a useful resource for anyone researching more on Zephyr vs Xray, but some information is no longer accurate:

1. Zephyr for Jira is now called Zephyr Squad (thanks @Harsh Dubey for flagging this)

2. All table items under Automation section should be marked as 'Yes' (please see here for reference)

3. Built-in rest api - yes 

4. Jenkins and bamboo integration - yes

5. Number of reports – it should be 5 reports and 9 dashboard gadgets 

6. Pricing has also changed. I'd recommend visiting the Atlassian Marketplace for both apps for most up to date info

 

Kind Regards,
Sergejs from Zephyr team

Like # people like this

Thanks @Sergejs Cuhrajs_SmartBear_

 

I can update the table as per your recommendation and will be needing your help as well. If you are available let's connect and discuss more.

Hi @Jayesh Wankhade

Sounds great, you can reach out to me via email sergejs.cuhrajs[at]smartbear[dot]com

 

Thanks,
Sergejs

Like Jayesh Wankhade likes this

Hi @Sergejs Cuhrajs_SmartBear_ how does REST API use test versioning? I am not seeing any reference when retrieving the test at Zephyr Scale for Jira Cloud API (smartbear.com)  (getTestCase endpoint) neither when updating it (I mean, does update increase test case version by default) at Zephyr Scale for Jira Cloud API (smartbear.com) (udpateTestCase endpoint)

Can the active test version be configured? I mean, working in a new test version but meanwhile it is commited still using current version 

Related to both topics:  ... Does the jenkins plugin always access to last version? or to active version? 

@Antonio Gil hey there, so Zephyr Scale doesn't support versioning within our REST API, so every update goes to the latest version.

Regarding the Jenkins plugin – yes it will always use the latest version (as it also uses the API).

Hope this helps!

Kind Regards,
Sergejs

Hi @Jayesh Wankhade ,

Would you please send me the updated table ?

Many thanks,

Sally

Like Jayesh Wankhade likes this

Hi @Jayesh Wankhade 

 

Sorry for jumping on the bandwagon here but are you able to send me the updated table as well? 

Thanks for the good work putting this together - its very helpful. 

Kieron

Like Jayesh Wankhade likes this

Hello All

Sorry for the delayed, the article is updated now with all new information along with pricing details as well.


Thanks for your support.

Jayesh

Like Kieron Southwood likes this
Like # people like this

Hi @Jayesh Wankhade 

 

Sorry for asking this, did you manage to also update the comparison table? Reading the comments it looks like the automation part it is the old version.

I am gathering information of differences and advantages between xray and zephyr. Which one do you prefer?

Like maree_milne-v likes this

Hi,

I am currently using Zephyr Squad and the performance is terrible. 

Is anyone using X-Ray? How is the experience?

@Deepa Akinapally I am trying both in the trial mode. What are the problems you are having with zephyr?

On my side: Visually I prefer Xray, but at the performance side I prefer Zephyr Squad. To write the test steps in a test case with Xray cloud is a pain. It take a couple of seconds to load it and increases a lot the time to do anything.

We currently have Zephyr Squad, updating scripts like adding/modifying/ deleting is painfully slow(couple seconds).

Adding Test Scripts to Cycle is very painful sometimes. Can't get the right cycle to show and can't navigate to next pages. 

Are you saying X-Ray is not any better?

@Deepa Akinapally 
The problems I am having with xray is during the creation of new Test case steps, test plan or test set. It takes 2-3 seconds for each interaction to fully open the issue details. It is frustrating sometimes and even take longer than 3s for specific cases

However, to edit or delete it is okish. Do you have a way to ask to the jira admin to put the trial version of xray? That would be good to compare.


In Zephyr Squad, I changed IssueType = Test to IssueType = Test Migrated. I lost all the Test Details.

I am shocked. There was no mention of the consequences of the action.

Then I changed it back to IssueType=Test, I still didn't get my Test Details back.

Did anyone experience the issue that I am facing? 

@Deepa AkinapallyIf you will change the issue type from 'Test' to any other issue type it will delete the test data. It's an expected behavior.
You can restore the backup (if you have) from the Admin -> General configuration section.

Regards,

Harsh

Comment

Log in or Sign up to comment
TAGS

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you