Latest Status of xTest in Testplan is incorrectly displayed

unknown

Hi guys,

I am a software developer at Deutsche Telekom AG and we use the xRay plugin in our team for our test automation and test management. Unfortunately, I noticed one thing that I can't explain at the moment.

But first of all, our procedure:

• We create xTests in Jira of the type Cucumber.
• We summarise these tests in test plans
• Our automation now uses the xRay API to read the test details, executes the tests and returns the Cucumber report to xRay via the corresponding interface.
• This creates an execution containing the current status of all test cases.
• We attach this execution to the test plan to see the current status.

Now we come to the issue that I cannot explain:

In the testplan you can see that the last status of some tests is "FAIL". When these tests are now opened, it can be seen that the last status of the execution is "PASS". On the testplan level, therefore, the most recent status is not displayed, but an incorrect status. On the screenshot you can see that the test case "WITAART-8501" has the status "FAIL" according to the testplan. As soon as it is expanded, you can see that the last execution status is "PASS".

I have already been able to observe this latency in one or two other test cases, but without any pattern, it seems that a wrong execution status is displayed in the test plan completely randomly.

Any ideas how we can solve this problem?

Rogério Paiva - Xray Xporter

Hi @[deleted]

Do you, by chance, use Test Environments?

If you do, then it should be related to that since Environments affect the calculation of the status.

And how do they affect the calculation?

The basic rule is that the Test only PASS if it has PASS in all environments. If you use at least one Environment, the Executions will be grouped by Environment and evaluated within the same Environment. For instance, if you have an execution in an Environment AAA, and it has failed, the final Test status will be FAIL, since, in environment AAA, it is FAIL, no matter if you have other Environments where it has PASS because it needs to PASS in all Environments.

Please check this documentation page (https://docs.getxray.app/display/XRAY/Working+with+Test+Environments) with additional details and examples on how the environments work with the calculation of the Status.

Any additional questions, please get in touch with Xray Support (http://xraysupport.getxray.app); we would gladly assist you.

Thank you.

Kind regards,
Rogério Paiva [Xray Support Team]