What is Traceability & How to use AIO Tests' Traceability Reports

[#1]

Traceability allows you to trace your requirements through a quality process ensuring you are delivering what was agreed upon in the requirements. It is a key element for managers to monitor throughout the project.

A project’s requirements need to be validated through testing. Traceability can be achieved by linking your requirements to test cases. This makes traceability a key performance indicator (KPI) of the project.

Below are the key benefits of maintaining traceability apart from it being a KPI.

1. Ensures the original requirements/goals have been covered by tests
2. Acts as a plan for QA to understand the scope of testing
3. Ability to show scope change impact on entire process
4. Understand testing progress

Traditionally, an Excel was used to maintain traceability as it was easy to read. But maintaining traceability through Excel is a very manual and time-consuming task.

Nowadays, almost all the test management tools offer some or the other kind of traceability matrix or traceability report as part of the system. AIO Tests offers a comprehensive view of traceability. Let’s explore it in detail.

AIO Tests – Test Management for Jira

All-In-One Tests (AIO Tests) is a simple and easy to use testing tool for Jira. It currently offers 2 traceability reports – Traceability Summary and Traceability Detail.

Traceability Summary: Summarized numbers related to coverage of requirements via cases that can be used for executive level reporting.

Traceability Detail:  Tabular format, a more traditional look, with details about requirements, cases, executions/runs and linked bugs information.

Let’s dig deeper into how to use these reports in AIO Tests.

Traceability Summary

 Let’s run down a few scenarios.

As a manager how will I know if all my requirements in upcoming release are covered for testing?

The input screen of AIO Tests’ Summary report is as below.  It lets one specify requirements via JQL (other options are a list of Jira issues, saved Jira Filters).

Include Child Issues: select this if full hierarchy of a Jira issue is required (Epic -> Story -> Task)

Cumulate requirements data: select this if information should be rolled up to the parent issue

Clicking on Generate with just the input of issue ids would return the number of tests against each issue and thus help in coverage analysis.

As a manager how will I know if all my requirements in upcoming release are covered, executed and if bugs have been found against them?

The Requirements section remains the same. But now let’s also specify the execution cycles in which cases of the specified requirements were run and check the box for retrieving any linked defects.

The merge strategy is incredibly unique and here is how it works.

Scenario 1: If you want to know the final execution status of your cases across cycles, use “Last Run”

Selecting “last run” takes the execution status of the case when it was last executed (latest execution) and so for above scenario, you will get pass against all 20 cases.

Scenario 2: If you want to determine the status of your case across different parameters, use “All Runs”

What “all runs” does is, it considers status of a case from all the cycles to determine the final status. In this case, after merging, the final status of case 1 will be failed.

If user specified “last run”, then only latest run result would be shown (in this case “Passed”) and it would be incorrect.

Output

The generated report has 3 sections

1. Summary number boxes giving coverage details
2. Charts showing case priority, execution status and defect status distribution
3. Tabular view of all requirement level coverage and execution details

Traceability Detail

Traceability Detail provides a more traditional – tabular/matrix – look of the traceability report showing the requirements, linked cases, execution of those cases and linked defects.

Output of Traceability Detail report

The report has 4 columns

1. Requirement – showing specified Jira requirements in a hierarchy
2. Cases – cases linked to these requirements
3. Test Execution Results – execution details (status, cycle, date of execution & who executed the case)
   • Include Only Last Run checked: for each case, only the latest run status is shown
   • Include Only Last Run not checked: for each case, status of all runs in each cycle will be shown
4. Defects – details of linked defects

The benefit of this report is that in one glance you see everything happening with a requirement. How many times a case has been run and what were the results. This information can be extremely helpful in decision making process.

-----

To summarize, Traceability reports are extremely helpful in understanding how your testing is going and if all your product requirements are being tested thoroughly. It helps QA Managers to understand the testing progress and how time is being spent and for project managers to understand which areas need more focus from a development perspective.

Without Traceability you can never be sure of the quality of your product.