nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Testing - How to run multiple test executions for same x-ray test

Tom Wood_tqvwdy

I have multiple tester based around the UK, testing at different times. I have set x-ray tests and created multiple test executions per test. My thought being that each tester would be able to complete a test execution in the hope that only once all the test executions have been set to "Pass" would the x-ray test then change to pass

Accepted answers

Stefan Salzl

From my point of view:

The testcase itself is "just" a description of "which steps needs to be done" to test a special feature/functionality.

The test execution is then an abstraction of the testcase for a special setting (eg. version --> I execute the steps from a testcase with application in version 1). The testcase then reports the "current" status.

So if the test execution with version 1 passes the testcase would say "green - passed". After deploying version 2 of the application, creating and executing a new test execution for version 2 that fails the test case will say "red - failed" because the current state is that this feature is (obviously) not working with the current version (2) proved by the test execution.

All comments

Stefan Salzl

From my point of view:

The testcase itself is "just" a description of "which steps needs to be done" to test a special feature/functionality.

Tom Wood_tqvwdy

Stefan - thanks this has given me food for thought on I approach this. Many thanks

Stefan Salzl

Awesome. Thanks for letting me know and really happy I could give a helpful input

ChrisK_i289hy

@Stefan Salzl is it correct that a Test Execution can only contain one instance of a (named) test? Therefore, to execute the same test a second time, you have to create a separate Test Execution?

If so, this solution does not fit the way in which the part of my organisation works. If I raise a defect, it is likely to be resolved in-house on an ad-hoc basis ('there and then'). In this scenario, I would prefer to re-run the test from the same execution, keeping the prior instance for historical record (of the defect). This would allow the Test Execution to have an overall status of Passed.

I guess I am stuck having to create multiple Test Executions, each with a separate Status, containing a different number of Tests.

I appreciate that this is not Agile and so Jira doesn't fit the expectation.
I could accept creating an entity such as a Test Plan or Test Set, having multiple Test Executions, utilising the Test Plan/Set to track the current status of all Tests, but it does not appear that Jira/Xray supports this approach either.

<<EDIT>> I have been working through this as I have been editing this response.
I have now found that Test Plan is split into two different views, Tests and Test Executions. It appears that Tests only shows the latest Status of each test (each test is only listed once in this view, being a cumulative view of all tests within the Test Executions), which is exactly what I need.

I would prefer to execute my tests from a single Test Set, but I guess having multiple Executions is more 'descriptive'. I will get used to it

suresh.m.mani

I have created test execution for round1 under test plan. Lets say total Test case count is 100, in which 50 TCs is passed and 50 TCs is failed.

I want to execute the failed test cases on the same JIRA key used for test execution round1, but we can't re run it with the same test execution JIRA key.

so we need to create another execution JIRA for next run.

If run with another JIRA for failed test cases, do we need to add the failed test cases or overall test set.?

If move the failed test case only for execution, how we will get a view of overall test cases status with already passed test cases.

ChrisK_i289hy

I think you are asking the same question as answered in my replies to this discussion. You need to create a Test Plan and add the Executions. This will give you the overall status.

NB it isn't that clever, if you edit the oldest Test (run), it will be seen as the latest update, and the status will be taken from that.
ie if Test (run) Execution-1 failed, Test (run) Execution-2 passed, Test Plan would correctly show overall status Test (run) as Passed, but if you edited Test (run) Execution-1 after this time (for whatever reason, not changing the result), the Plan would show the overall status of Test (run) as Failed.

If you use something similar to the suggested naming convention, you can see the Test Executions of each Test, and the name will give you a 'quick view' of when you ran it.

BTW You posted an Answer to the initial question. You need to post a Reply to 'someone' if you are asking a question.

Jan-Hendrik Rolf

Hello!

When do I have to use existings test executions and when do I use new test executions?

ChrisK_i289hy

I am not a Jira or xray expert, but I believe your question is answered by considering that the tests (consider these a single instance to be executed, ie runs) within the Execution, can only have 'one result'. This also includes each step only having one Actual Result. Therefore, if you 're-use' the existing execution, you will overwrite a previous result.

Personally, I much prefer ALM, in which I can simply run the test again, creating another instance (seeing a history of those test runs within the Test Lab).

Anyway, to have distinct runs of a test (ie with its own results), you would need a separate executions.

NB Having all these executions is the reason for the suggested naming convention, as you can quickly select the execution by viewing Test Runs from within the actual xray test (being a selectable 'view' from Test details). exmmyy - 'description' is fine for me, but you might prefer exddmmyy depending on how may iterations/runs you expect.

Hope that helps.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

xray

jira

atl-gruland

test

user-acceptance-test