Test writing in QAlity Plus vs Xray vs Zephyr

[#1]

Since testing is a crucial part of software development, there are numerous applications designed to assist in this process. On Atlassian Marketplace alone you will find over 200 testing apps for Jira available. It can get overwhelming, trying to find the most suitable solution for you. Testing apps differ not only in terms of functionality and price but also the user interface. Some apps might have all the features you need and still be hard to learn if their design does not resonate with you. Others might be great in all aspects but the price. To help you navigate through these complex differences, and assist you in finding the right option for you, we’ve compared 3 popular testing apps.

The focus of this article will be test writing features offered by QAlity Plus, Xray, and Zephyr Squad. We’ll be considering functionality, price, and design differences to determine what makes each of these applications unique, and help you decide which one suits you best.

Creating tests

The feature that lies at the heart of testing is test creation. Naturally, each of the mentioned apps is equipped with this functionality. However, the details are approached differently by different vendors. 

The most standard way to write tests in Jira apps is by selecting a test issue type while creating an issue. It will result in creating an issue specially dedicated to testing. The option is available in all 3 apps.

Additionally, in QAlity Plus the user can also create test cases directly in already existing Jira issues. 

 

By clicking the Add Test Case button inside an issue, the user can create a new test case with the default name equal to the Jira issue they're working on. The test case will be linked to the issue. It can be edited both in the issue that it was created in and in the test issue itself. 

From inside the issue you can create multiple test cases, and each one of them will automatically add a new test issue to your Jira. It’s a handy shortcut if you want to quickly create a bunch of test cases related to one issue.

Creating test steps

QAlity Plus

After creating a test case, your cursor will be automatically focused on the first step’s input. Once you insert the required information and accept it, the next step will be automatically created. 

You can also click the Add new Test Step button at the bottom of the test case or click the test case menu, and then select Add new Test Step. Both options will result in adding a test step at the bottom of your test steps list.

Xray

You can create test steps in two ways. By clicking on the Create Step button at the top right corner of your test issue, you will add a step at the bottom of your test steps list. By clicking on the Create Step button in between already existing test steps, you will create a new one in the desired place on your list.

Zephyr Squad

You can add test steps by clicking on the button Add Step at the top right corner of your test case. A new step will be added at the end of your test steps list.

Viewing options

Depending on the situation, you might want to view more or less detailed information on your test cases. Testing applications offer different viewing options.

QAlity Plus

Simple view

Displays only test steps and their expected results, to allow clear and quick insight into the actions that are performed, and their desired outcome.

Detailed view

Includes test steps, data, results, and attachments, covering all of the crucial information on each test step.

Collapse/Expand test cases

You can quickly expand all test cases to see their details, or collapse them to hide the specifics when not needed.

You can also expand or collapse a particular step by clicking on the icon wrapper on the left of the test case.

Xray

Grid view

Designed to give the user a quick overview of the most important information in a compact, shortened form.

List view

More detailed view, displaying all the information on each of the test steps.

Collapse/Expand test cases

You can collapse or expand test cases by clicking on one of the icons at the top of your test case. 

Specific test steps can also be collapsed or expanded. To achieve this, you need to click on the collapse/expand icon in the menu on the right side of your test step.

Zephyr Squad

Lite view

Displays only test steps, test data, and test result columns. Designed for a quick overview.

Detail view

 

Allows adding additional columns and customizing the elements that are to be shown. 

Editing

Once created, test cases and test steps need to be easily editable. Editing features are slightly different in each of the apps.

QAlity Plus

All elements of the test case and test steps can be edited inline. It means that you can edit these elements by simply clicking on them.

Xray

When it comes to test type and definition, you can modify the information using the inline editing feature. 

In order to change something in your test step, you need to enter an edit mode:

Zephyr Squad

All of the information can be edited in a special edit mode, similar to the one used in Jira issue creation.

Reordering

All 3 applications offer an intuitive, user-friendly drag-and-drop feature, allowing the user to change the order of test steps by simply repositioning them on the list.

Additionally, in Xray you can manually set the rank of a certain test step to determine its exact position on the list.

 

Cloning and deleting

QAlity Plus

You can clone your test cases by clicking on the three-dot menu at the right top corner of your test issue and choosing Clone. The test case will be copied along with all of the details and test steps.

You can also clone each test step separately. It will result in a new step appearing under the one that was cloned.

Similarly, you can delete test cases or test steps by choosing the Delete option in the three-dot menu.

Xray

You can find cloning and deleting options in the test issue menu.

 

To clone or delete test steps, you need to click the corresponding icon in the test case menu on the right. The cloned step will appear under the one that was cloned.

Zephyr Squad

You can clone or delete a test case just like any other issue, by choosing the right option in the test issue menu.

To clone or delete a test step, you need to click on the corresponding icon from the Action column in your test steps list. While cloning a test step, you’ll get to decide on what position on your list you want it to be placed.

 

Cloud Pricing

The aspect that can not be omitted in the process of choosing the right solution is pricing. All of the 3 apps compared in this article are equipped with crucial features needed for quick and comfortable test writing. Some of them offer additional functionalities, such as defining preconditions or test types in Xray. However, wider functionality is naturally reflected in the higher price. Depending on what your needs are, more costly options might be either a great investment or a waste of money.

Here are the monthly cloud prices for each of the three apps, shown in three examples - for up to 10 users, for 50 users, and for 1000 users:

QAlity Plus   

Up to 10 users free

50 users = 75.50$/mo

1000 users = 570$/mo 

Xray 

10 users = 10$/mo

50 users = 212.50$/mo

1000 users = 1,362.50$/mo

Zephyr Squad

Up to 10 users free

50 users = 255.50$/mo

1000 users = 1,874.50$/mo

Summary

The above comparison can serve as a reference for considering the available options of testing apps, their unique features and user interfaces, as well as their cost. It is focused on the test writing-related functionalities only, with the purpose of zooming in on the details and nuances of each vendor’s offer. In the following articles, we will compare other areas of testing apps’ features, such as test execution and reporting, to provide a full picture and, hopefully, make it easier for you to decide on the solution most suitable for your needs.