How to reset the 'TestRunStatus' field values that are not synchonized with the real 'Test Execution' status ?

Hi there,

My XRay is in 2.0.3 version, 'Final statuses have precedence over non-final statuses' is not activated and 'Test Run Status Custom Field Version' is set to 'No Version'

 

My problem is that, since we've updated from v1 to v2 (10th of October), some of the projects are experiencing problems with their TestRunStatuses... The last TestRunStatus is still the one from the v1 !

 

You can see the results in this pic : 

TestRunStatus_vs_TestExecutionResults.jpg

In the left, we see the result from a jql search in whitch we'd must have the last executed test run status, that are actually all PASS...

 

Here's what I've tried on another JIRA platform (copy from prod), without any success :

  • Reindex... Mutiple times
  • Activate / Deactivate 'Final statuses have precedence over non-final statuses'
  • Integrity Checker for ALL... Multiple times
  • Clear Test Run Status Custom Field for only 1 test withissue menu, and reindex
  • Clear Test Run Status Custom Field for the 4 precited tests with bulk change, and reindex
  • Clear Test Run Status Custom Field for ALL the tests with admin module, and reindex
  • Upgrade from 2.0.2 to 2.0.3 (I saw this fix so I tried : XRAY-1130)

     

I've no more idea. Can you help me ?

Is it a bug ?

In this case, will it be fix and when ?

 

Thanks in advance !

VM

2 answers

Hi Vincent,

 

I tried to reproduce this behaviour using the same configurations but the TestRunStatus is always correct.

The only thing that I can remember is Environments. Starting from version 2.0 Xray now provides the concept of multiple executions of the same Test by environments (e.g IOS, Android). For this, the Test Environments Custom Field must be populated in Test Execution issues.

The Environments will directly affect the TestRunStatus calculation. If for instance you have a Test PASSED for Android and FAILED for IOS, the aggregated result for the Test will be FAILED. If you have another Test Execution for the same version with empty environment, then Xray will consider this as another environment as well.

 

If this is not the case, please open a ticket in our support desk at http://xraysupport.xpand-it.com and will try to debug the problem.

 

Kind Regards,

Bruno Conde.

Hi Bruno and thank you.

Actualy, we don't use the 'Test Environments' field...

But I saw this in the 'Integrity Checker' documentation :

Starting from version 2.0.0 of Xray for JIRA, the following Xray custom fields are calculated upon any change. This means the value for the custom fields is stored instead of calculated on the fly. This improves performance considerably for several Xray screens and queries.

/[...]

Despite our best effort, there might be certain use cases where the custom fields are not updated, resulting in out of date values. When this happens we provide the ability to remove the value of all custom fields in this page. The value will then be calculated upon fist access, or when a JIRA re-index operation is performed.

 

I've done a full clear TestRunStatus cache and a full JIRA reindex... Same result... sad

I'll log a ticket to your support.

Thank you.

Best regards.

VM

 

Hi Vincent,

 

Just to clarify, the TestRunStatus value can be invalid only in certain corner cases. This is why we provide these integrity check actions.

Even if the custom field value is invalid, reseting the value should immediately update the custom field with the correct value. Because you already performed these reset actions, the problem must be in the calculation itself.

 

Best Regards,

Bruno Conde.

Suggest an answer

Log in or Sign up to answer
Atlassian Community Anniversary

Happy Anniversary, Atlassian Community!

This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.

Read more
Community showcase
Julia Dillon
Posted Tuesday in Jira

Tell us how your team runs on Jira!

Hey Atlassian Community! Today we are launching a bunch of customer stories about the amazing work teams, like Dropbox and Twilio, are doing with Jira. You can check out the stories here. The thi...

184 views 1 17
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you