Comparison between Zephyr & Xray

[#1]

I am sharing this article with both Zephyr and Xray information and everything should be in one place. It will be easy to know when selecting Zephyr or Xray.

Zephyr and Xray are both Jira plug-ins used for test case management i.e., test case writing and execution purposes. Both do almost similar kinds of work with some advantages and disadvantages of them with respect to cost, automation, reporting, etc. 

Here is the feature level comparison for Zephyr and Xray

Reports:

Zephyr provides Traceability Report, Test Case Execution Report, Test Execution Burndown Chart and Top Defects Impacting Testing, etc. whereas Xray provides Traceability Report, Overall Requirement Coverage Report, Historical Requirement Coverage Report, Test Plans Report, Test Executions Report, Test Runs Report, etc.

Cost:

Reference Links:

Features: https://interworks.com.mk/comparison-of-jira-test-management-plugins/

Zephyr Advantage: https://www.getzephyr.com/insights/why-zephyr-jira-1-alternative-xray

Zephyr Details: https://zephyrdocs.atlassian.net/wiki/spaces/ZFJ0300/overview

Xray Advantage: https://www.getxray.app/xray-vs-zephyr/

Xray Details: https://confluence.xpand-it.com/display/public/XRAY/About+Xray

Zephyr Pricing: https://marketplace.atlassian.com/apps/1014681/zephyr-for-jira-test-management?hosting=server&tab=pricing

Xray Pricing: https://marketplace.atlassian.com/apps/1211769/xray-test-management-for-jira?hosting=server&tab=pricing

[Christian Pascher]

Regarding versioning:

The approach Xray is using is not really versioning that people expect. You cannot just select that you want to include version 2 into one test execution and version 3 of the same test in another test execution. Versioning is only supported in the way Jira is doing it.... But you definitely get a frozen list of test steps in a test execution that won't be changed if you change the test case afterwards. Which is most of the time enough, also for enterprises and big projects/programs from my experience. See https://docs.getxray.app/display/XRAY/Test+Versioning

Real versioning of test cases is only possible if you don't use Jira issues to store the date like it is done in for example in QMetry Test Management for Jira (https://marketplace.atlassian.com/apps/1215144/qmetry-test-management-for-jira?hosting=cloud&tab=overview)

[Taranjeet Singh]

This is very useful comparison of Xray and Zepyhr apps. Thanks for sharing this @Jayesh Wankhade !

[Jayesh Wankhade]

Gald that you liked it @Taranjeet Singh and helpful for you. Thanks for motivating me. 

[Piotr]

Hi, @Jayesh Wankhade  could you send me the snapshots from 'Zephyr vs Xray" comparison?

Unfortunately, I can't correctly displays Features 1 and 2 you posted.

 email: pwilkowski@o2.pl

[Jayesh Wankhade]

Thanks @Piotr piotr for pointing it out, and sorry for the late response. I have updated the images here as well and sent you on given email. Please let me know if you received it not.

[Piotr]

Thanks @Jayesh Wankhade for sharing your tool comparison sheet!

[Harsh Dubey]

@Jayesh Wankhade @Piotr This is outdated feature comparison list. ZSquad (brand name got updated as well) offers all the test management, automation related features.

Attaching the link for same: Feature-Link 

[Jayesh Wankhade]

Thanks, @Harsh Dubey for highlighting it. This blog was written a year back. 

[Sergejs Cuhrajs]

Hi @Jayesh Wankhade

Would you mind updating the table? I think this is a useful resource for anyone researching more on Zephyr vs Xray, but some information is no longer accurate:

1. Zephyr for Jira is now called Zephyr Squad (thanks @Harsh Dubey for flagging this)

2. All table items under Automation section should be marked as 'Yes' (please see here for reference)

3. Built-in rest api - yes 

4. Jenkins and bamboo integration - yes

5. Number of reports – it should be 5 reports and 9 dashboard gadgets 

6. Pricing has also changed. I'd recommend visiting the Atlassian Marketplace for both apps for most up to date info

Kind Regards,
Sergejs from Zephyr team

[Jayesh Wankhade]

Thanks @Sergejs Cuhrajs

I can update the table as per your recommendation and will be needing your help as well. If you are available let's connect and discuss more.

[Sergejs Cuhrajs]

Hi @Jayesh Wankhade

Sounds great, you can reach out to me via email sergejs.cuhrajs[at]smartbear[dot]com

Thanks,
Sergejs

[Jayesh Wankhade]

Thanks @Sergejs Cuhrajs 

[Antonio Gil]

Hi @Sergejs Cuhrajs how does REST API use test versioning? I am not seeing any reference when retrieving the test at Zephyr Scale for Jira Cloud API (smartbear.com)  (getTestCase endpoint) neither when updating it (I mean, does update increase test case version by default) at Zephyr Scale for Jira Cloud API (smartbear.com) (udpateTestCase endpoint)

Can the active test version be configured? I mean, working in a new test version but meanwhile it is commited still using current version 

Related to both topics:  ... Does the jenkins plugin always access to last version? or to active version? 

[Sergejs Cuhrajs]

@Antonio Gil hey there, so Zephyr Scale doesn't support versioning within our REST API, so every update goes to the latest version.

Regarding the Jenkins plugin – yes it will always use the latest version (as it also uses the API).

Hope this helps!

Kind Regards,
Sergejs