Test cycles in QAlity Plus vs Xray vs Zephyr

[#1]

In the previous article, we discussed test writing features in 3 popular apps for Jira - QAlity Plus, Xray, and Zephyr Squad. We’ve compared their functionality as well as the details of user interfaces. Having covered the most important aspects of test creation, we’re now moving on to test case grouping and organization.

It would be pointless to discuss testing without mentioning test cycles. An effective testing process requires not only skills and knowledge but also a well-organized environment. To make it easier for the user, testing apps usually provide features that enable sorting test cases into test cycles - sets of test cases focused on achieving certain results in a certain period of time.

In this section, we will compare test cycle-related features available in QAlity Plus, Xray, and Zephyr.

Overview

In all 3 apps, you will find a page dedicated to test cycles, providing a general overview of your work. These pages, however, differ when it comes to viewing and management options, allowing different actions to be taken.

QAlity Plus

The test Cycle Overview page offers an insight into your test cycles along with the basic information on them: project, version it is assigned to, the status of test executions, creation date, and due date. Test cycles can be filtered by project and by version.

By clicking on the title of a test case, you will be taken to the page dedicated to the chosen test cycle, where all the details are gathered: project, version, description, due date, test cases, and status.

You can execute or delete related test cases directly from this page.

Xray

Test Plan

Instead of test cycles, there is a feature called Test Plan. It provides an overview of tests included in your Test Plan along with their executions.

The Test Plan table offers a handful of information such as a list of tests, their executions, the dataset, the latest status as well as the overall execution status. By clicking on the list icon (marked by no. 6) you will be transferred to a page dedicated to test runs of the selected test.

 

Test Plan can be also viewed on the Test Plan Board - a more complex and detailed alternative to the Test Plan issue view.

Test Set & Test Repository

Additionally, there are 2 different types of test grouping available: Test Set and Test Repository. 

Test Set is a simple flat list of tests.

Test Repository is a more advanced model. It provides a hierarchical tree-like structure and allows organizing tests within folders and sub-folders. 

Zephyr Squad

On the Test Cycle Summary page, you will find a detailed account of your test cycles. The contents of the page will change based on the version of the project that you choose to display. The basic information in your Cycle Summary includes status, summary, start date, end date, and more.

 

Similarly to the test steps discussed in the previous article, you can customize the view by selecting columns that you would like to be displayed in your Cycle Summary. From this page, you can also view test cycles details or execute your tests by clicking on them.

Tests within test cycles can be organized in folders. Each folder can have its own executions.

Creating test cycles

QAlity Plus

There are two ways to create a new test cycle:

• Choose the Create Test Cycle option from the QAlity menu on the left.
• Open the Test Cycles Overview and click the Create Test Cycle button at the top right corner.

Either option will result in opening a window where you can write down the details of your new test cycle and the project it’s assigned to. You can optionally provide the version, description, and due date. 

Xray

To create a Test Plan you need to choose the Test Plan issue type in the issue creation screen. You can enter the details such as components, description, and version.

Zephyr Squad

You can create a new test cycle by clicking on the Create New Test Cycle button in the Test Cycles menu.

A window will appear, allowing you to enter the details:

Adding tests to test cycles

QAlity Plus

There are several ways to add test cases to your test cycles.

From the Test Cycle context menu within your Jira issue:

 

From the Test Cycle page:

 

You can also bulk-add multiple test cases. To do that, you need to go to the Test Cases Overview page, use the JQL search or apply the existing filter, and click the QAlity Plus action button.

You will be asked to choose the project and the test cycle, and then confirm whether all of the automatically selected issues are to be added to the test cycle:

Xray

You can add tests to your test plan either in an issue creation screen or from your test plan table. 

 

Once you click on the Add Tests button, and choose your preferred option, a window will appear, allowing you to select or search tests that you want to add to your test plan.

 

Zephyr Squad

There are multiple ways of adding tests to your test cycles.

When viewing a Test:

 

When viewing a Test Cycle:

 

Individually:

 

Via a saved search filter:

 

From another Test Cycle:

Via a JQL query:

 

Reordering test cases within test cycles

QAlity Plus

You can drag & drop test cases to change their positioning within a test cycle.

Xray

You can change the order of tests within your Test Plan either by dragging & dropping or by assigning a new rank to them.

 

Cloud Pricing

Pricing is one of the crucial factors influencing the choice of an application. The 3 plugins discussed in this article are representing different financial categories, allowing each user to find a solution optimal for their needs and capabilities.

By comparing QAlity Plus, Xray and Zephyr Squad features related to test cycles, you will notice how differently each of the apps approaches the topic. For example, simplistic QAlity Plus stands in high contrast with the complexity of Xray. All 3 apps have their own unique assets. They differ not only in terms of the user interface but also in terms of the whole organization of tests and test cases. 

Depending on the user’s needs, different apps will be viable and worth considering. More detailed test grouping and organizing options are tied to higher costs, and require a lot of time and effort to learn. However, in the case of complex testing plans and operations, they can bring a lot of value to the project. On the other hand, simpler solutions don’t offer as many features, and in some cases might end up being not elaborate enough to meet the requirements. However, their straightforward design and reduced complexity can also serve as an advantage, making them easy to use and quick to learn. Another advantage is the lower price.

Here are the monthly cloud prices for each of the three apps, shown in three examples - for up to 10 users, for 50 users, and for 1000 users:

QAlity Plus   

Up to 10 users free

50 users = 75.50$/mo

1000 users = 570$/mo

Xray 

10 users = 10$/mo

50 users = 212.50$/mo

1000 users = 1,362.50$/mo

Zephyr Squad

Up to 10 users free

50 users = 255.50$/mo

1000 users = 1,874.50$/mo

Summary

The above comparison can serve as a reference for considering the available options for testing apps, their unique features and user interfaces, as well as their cost. It is focused on the test cycles and test organizing-related functionalities only, with the purpose of zooming in on the details and nuances of each vendor’s offer. In the following articles, we will compare other areas of testing apps’ features, such as test execution and reporting, to provide a full picture and, hopefully, make it easier for you to decide on the solution most suitable for your needs.