XRay Import Execution Results - Exception when importing test execution result via REST API

[Thibault Jamard]

Hi,

I encountered a problem when I try to import test result using Xray REST API v1.

The problem occurs at the update to Xray 5.1.0. It seems like the endpoint "/import/execution" is not working anymore.

I got the following error message :

{"error":"An exception occurred while handling a v2 REST API request: java.lang.NullPointerException"}

Step to reproduce :

Execute the following command (using API v2 now) :

curl -X POST "https://my.jira.com/rest/raven/2.0/api/import/execution" -u user:pwd -H "accept: application/json" -H "Content-Type: application/json" -d @data.json 

with data.json :

{
  "testExecutionKey":"SBX-2820",
  "tests":[
    {
       "testKey":"SBX-2821",
       "status":"FAIL",
       "comment":"it does not work"
    },
    {
      "testKey":"SBX-2824",
      "status":"FAIL",
      "comment":"it does not work too"
    }
  ]
}

Despite the error, the request seems to be seen by the test execution, as it logs a new line in the History of the issue. Furthermore, when I use the "info" object instead of the "testExecutionKey" field : the new test execution is well created, but tests are not added in it (nor their results).

Note : I don't know which Xray version I was using when it was working, but I don't see any compatibility break with the new version

Any help is appreciated :)

[Paul FLYE SAINTE MARIE]

Xray Team have solved the problem in a newest version who will come out quickly (perhaps today or next week) - Version 5..1.1 - https://jira.getxray.app/browse/XRAY-8086.

[Barak Maoz]

Hey for me It's still not working, I still have the same error with 

"An exception occurred while handling a v2 REST API request: java.lang.NullPointerException"

for now, there is any way to make it work without your fix? 

[Paul FLYE SAINTE MARIE]

Hello @Barak Maoz : You right :(
The ticket https://jira.getxray.app/browse/XRAY-8086 was not delivered with the version 5.1.1.

So we need to wait the version 5.1.2 with this ticket included...

I asked to Xpand (Xray editor) when they will delivered this newest version.

I hope soon.

You can contact them also or vote to this ticket to be included as fast as possible.

[Rogério Paiva - Xray Xporter]

Hi,

Version 5.1.1 should fix the defect XRAY-8086.

If anyone is still experiencing some issues after the upgrade. It is best to contact Xray Support (http://xraysupport.getxray.app) to share your configuration so that the support team can troubleshoot the cause since it might be a new situation.

Thank you.

Kind regards,
Rogério Paiva [Xray Support Team]

[Paul FLYE SAINTE MARIE]

Hello,

We have the same bug as you on our Jira software instance with Xray 5.1.0.

We contact Xray editor to investigate with them.

I think it's a regression on the Xray version 5.1.0 sadly :(
I advice you to downgrade your Xray on version 5.0.5.
It's corrected inside this version.

You can open a ticket on their support here : https://jira.getxray.app/servicedesk/customer/portal/2

And you can ask them to link this ticket to our ticket : SUPPORT-42269.

[Thibault Jamard]

Thanks for your reply

We will downgrade our Xray version, waiting for a fix. I also created a support ticket and ask them to link it with yours.

Note : if you are interested, I found a workaround using the endpoints :

- GET /testrun to get the testrun ID

- PUT /testrun/{id} to push result to that id

but this requires to link the test to the test execution before running tests

[Yilmaz Kocak]

if you set Test Type under Test Details like (Automated) in your testcases, the problems should be solved. Until 5.0.5 Xray version it was ignored. 

[Thibault Jamard]

Hello,

Thanks for the answer, but I'm not sure to understand.

Do you mean that a new field like testType becomes mandatory with Xray 5.1.0 ? If yes, it is not documented anywhere (or I didn't find it)
I tried to add a field in the json :

{"testExecutionKey": "SBX-2823" ,"tests": [{"testKey": "SBX-2821", "status": "FAIL", "testType": "Manual"}]}

but it doesn't work : "testType do not follow the correct format"

All my tests are with the type Manual. Automated is not one of the suggestion, I've got ['Manual', 'Generic', 'Cucumber']. That is why I don't know what you meant

[Yilmaz Kocak]

We have over 400 automated cucumber test cases. We had never issue until 5.1.1 upgrade. With 5.1.1 version of XRAY we were not able to get the test results over REST API. 

Our team analyzed the issue together with Xpand IT. They delivered an hotfix for us. But we did not install the hotfix, because we understood where the problem is. We just set for our testcases the test type where it as empty and the problem was solved for us.

You should ad the testtype in XRAY not over REST API, if you have the test case 

SBX-2821

in XRAY.

If yo have support with Xpand IT, you should check it with them.

 Hint: Our cucumber test scripts are in code base. In xray we have only the dummy test cases (place holders) to link them with the requirements. Test executions and test results are getting over REST API. 

[Barak Maoz]

where did you add this testType? Do I need to add the test type to the Json?
because in the Xray all my tests are on testType= Manual. 
if you can send a screenshot with the change that you did to make it work again it will be great
Thank you!

[Yilmaz Kocak]

HI Barak

No, not in Json

In our case there were some test cases in the test exectuion where some of them did not have test type after we set the test type our issue was solved.

[Thibault Jamard]

Hi,

I don't have the test type : Automated. I can only choose between ['Manual', 'Generic', 'Cucumber']. In your screenshot, I assume that the test type 'Automated' is an equivalent of 'Generic', as you have the "Generic Test Definition"

I set my test SBX-2821 to the 3 types, but I reproduced the bug with all. I don't have any test without the test type set to None, it is not even possible as it is a mandatory field for us.

[Rogério Paiva - Xray Xporter]

Hi @Thibault Jamard 

It is best to contact Xray Support (http://xraysupport.getxray.app) to share your configuration so that the support team can troubleshoot the cause.

Thank you.

Kind regards,
Rogério Paiva [Xray Support Team]

[Thibault Jamard]

Thank you @Rogério Paiva - Xray Xporter 

I was waiting the new version with the fix to install it. I see it is available now.

If I still see issue with the new version, I won't hesitate to contact Xray support