Xray reporting, Test Coverage using Tasks and Test Plans

ChrisK_i289hy

Hi,

I believe I can use several different issue types as my 'requirement'.  I have no need for Epics and Stories, and so a simple Task suits my purposes.

I have many tests in xRay.  I may or may not choose to use Test Sets.  Obviously, to run them, I need an Execution.  For re-testing, I need additional executions, and so I use Test Plan to keep track of the most recent result.

To my mind, I need only link (Is tested by) the Test Plans that fulfil my 'requirement' to a Task...

I have changed my App>Xray Settings>Test Coverage>Coverable Issue Types to include Test Plan and Task.

I have Issue links relation as Testing, Issue Link Type Direction as Inward - Is being tested by

NOTE: I am doing this retrospectively, ie AFTER I have completed my Testing (test runs executed).  For future Jira management, I am now investigating reporting.

I have created a Task called Functional Testing

I have linked three Test Plans as Is tested by

When I select Test Coverage, the window shows UNCOVERED ???

Surely the whole point of issue linking is to fulfil the purpose of adding an issue that has tests?

 

NB If I open one of the Test Plans, because I have xRay Settings including Test Plan as a Coverable Item, it too has Test Coverage. but this too has UNCOVERED with a button to Add Tests ... DESPITE DIRECTLY BELOW THIS, there is an Overall Execution Status of 8 PASSED.

 

My first question is, can I add Test Plans to a Task with the expectation that coverage of the Task will report on the test status of all Test Plans?

My second, how do I do this?

 

I am wondering if xray have not integrated Test Coverage very well, and it will only report on tests that have been explicitly added as a Test Coverage tests.  In the case of a Test Plan, this doesn't look like it makes any sense, as it appears to be asking to add individual tests to the Plan under/within Test Coverage (ie not tests in an Execution, or more to the point, multiple Executions, being the reason to have the plan)

ChrisK_i289hy

This is not an answer, just an update ...why is Answer the only way to respond to a question here??

Anyway, I have attempted to investigate further, unfortunately making my project a mess (hopefully the PM will not spot it)

On the basis that I am expecting the Test Plans to provide the test coverage for the Task, I have viewed/opened one of the three Test Plans and chosen to Add Test under Test Coverage.
As I had guessed, this lists the newly added test under Test Coverage, NOT UNDER Tests, ie the tests within the PLAN.
If I select this newly added test and choose Execute > With Test Execution, it creates a Test Execution that is NOT IN THE PLAN!!!!
Worse still, now I have added this test, there appears to be NO WAY TO REMOVE IT ?!?!?!  There are no three dots/Action as there are for each test under the plans Tests.

Surely I can choose to remove a test from coverage that I have added?

...however, if I manually add the Test Execution to the plan, then choose Test Coverage, Analysis & Scope > Scope - Test Plan > Test Plan - 'the one I am in!', I get NOTRUN.

This implies that I cannot use existing Test Executions within a plan, ie to report retrospectively.  It also implies that I have to add all tests 'individually' to Test Coverage, and to create Test Executions as I go along.  This is unintuitive and will cause confusion.  Also, the Test Plan layout will be a mess, having each test listed twice, under Test Coverage and Tests.

That all said, if I go back to my (linked) Task, Test Coverage IS STILL SHOWING UNCOVERED.

If I were to plan this right from the beginning, it looks like I would create a Task and add each test, choosing to create a Test Execution as I go along (for each set of tests that I want in a separate Execution).  If I want to re-test, I guess I have to create another Execution FROM THE TASK ... I guess I then create a Test Plan to keep track of the status of the two Executions, but it looks like this is entirely independent of the Task, so it appears that Issue Links relation has no purpose in this scenario.

Hopefully someone understands what I am trying to do, and can tell me if it is possible.  At the moment, my perception is that xray coverage is very poorly implemented.