Jira Dashboards - Not Showing the latest test result

[Jamie Mcdevitt]

We are using Jira Server. Tests and managed by test sets/plans and run via a test execution. In my example; lets say I have a test execution: 'Sprint 1' that has test x with a status: PASSED and then later I have a new test execution: 'Sprint 2' for the same test plan and now test x has a status: 'BLOCKED' or 'TODO'.

I would like to have a dashboard showing the latest test result, however when pulling the test or the test plan it is showing test x with a status of 'PASSED' which is not accurate.

In my original example, if test x is set to 'FAILED' in 'Sprint 2' then the status IS reflected. I can see that test x is 'FAILED' when viewing a dashboard showing either the test or test plan state. This seems inconsistent, is it a bug?

Edit: we are using XRAY plugin as a test management tool

[Gaurav]

Hello @Jamie Mcdevitt 

Welcome to the community!!
The difference in the status of issues on the dashboard could be due to the reason that the gadgets are not set to refresh after a defined duration of time.

This can be done by the owner of the board. Kindly connect with them for this.

Please let me know if you would need the steps for this.

Kindly accept the answer, if this helps resolve the query.

[Jamie Mcdevitt]

Hi @Gaurav, thank you for your reply however this is not the issue. I have already enabled the gadgets to refresh every 15 minutes but the status remains the same. The issue is not solely with the dashboard, If I open up the testplan the 'Overall Execution Status' reflects the old value. It is specific to the states: 'TODO' and 'BLOCKED'.

If a test case within an OLD test execution has a test result status and later rerun as part of a new test execution, the 'Overall Execution Status' will always choose the older state if the new state is 'TODO' or 'BLOCKED', if any other state is chosen then it will reflect it accurately. This almost seems like its done by design but its frustrating and feels more like a bug.

[Gaurav]

Hello @Jamie Mcdevitt 

It seems there is a status mismatch, and the test execution reflects the OLD run status and not the NEW run status.

In my limited experience, we need to create new Test Plans and add the Test Cases prior to execution. I would suggest that you can try that and see if that helps (if not doing it already :))

[Jamie Mcdevitt]

@Gaurav per your suggestion, are you saying for each rerun of a test case, we should be creating new test plans every time? Is that not the purpose of the test execution?

To confirm I will go through an example:

Test plan: SERVICE_X_SMOKE_TESTS - Contains 10 test cases

Test Execution: SPRINT_1_SERVICE_X_SMOKE_TESTS - Result: 8 pass 2 fail

Team reviews results, decides they will not fix the 2 failed tests in sprint 1, agrees to block the tests.

Test Execution: SPRINT_2_SERVICE_X_SMOKE_TESTS - Result: 8 pass 2 blocked

Test plan: SERVICE_X_SMOKE_TESTS - Now reflects two test executions linked, clicking into each execution reflects the results correctly but the 'Overall Execution Status' of the test plan reflects 8 passed 2 fail

The test plan is now not reflecting that the team has blocked 2 tests and that in the latest execution linked their result is indeed blocked. Per your suggestion instead of this approach the team needs to create a new test plan EVERY time they wish to run an execution for truly accurate results? This seems counterproductive to how test plans/test executions work and defeats the purpose of linking executions to plans.

Additionally to this, if the result is ALWAYS pass/fail then as I stated before the 'Overall Execution Status' is correct and reflects the delta correctly. It just IGNORES blocked/todo statuses and appears to be a bug?

[Roman Mulladzhanov]

Im having the same issue. Was there ever a viable solution to this question that didnt require creating anew test plan?