How to import test execution results using REST API and XRay for JIRA

[Jamie Atkins]

Hello,

I am trying to import test execution results for my project but am having difficulty associating tests with the test execution using the REST API. I have tried to emulate the instructions found at http://xray.xpand-it.com/display/XRAY/REST+API but it does not seem to be working. I can import a test execution without any associated tests.

Here is my request which I am posting to http://<URL>//rest/api/2/issue

"{""fields"": {""project"":{""key"": ""ET""}, ""summary"": ""Test execution"", ""description"":""Description"", ""issuetype"":{""id"":""10302""}, ""tests"":[{""testKey"" : ""ET-879"",""comment"" : ""Successful execution"",""status"" : ""PASS""},{""testKey"" : ""ET-880"",""comment"" : ""Execution failed"",""status"" : ""FAIL""},{""testKey"" : ""ET-881"",""comment"" : ""Successful execution"",""status"" : ""PASS""}]}}"

The "tests" field is not recognised, but I can't find the appropriate replacement to get this to work.

Any help you can give would be appreciated.

Thanks

Jamie

[Bruno Conde · Accepted solution]

Hi Jamie,

You must use the REST path specific to Xray: /rest/raven/1.0/import/execution

The JSON you are using is for the default REST service of JIRA. This will just create a Test Execution issue. 

If you follow the instructions on http://xray.xpand-it.com/display/XRAY/REST+API, it should be able to import the execution results by creating a new Test Execution issue, if you specify the "info" element (this will create a Test Execution issue in the same project where the Test issues are but we will soon release a new version where you can specify the project explicitly). You can update an existing test Execution if, instead of the "info" element you pass the "testExecutionKey": http://xray.xpand-it.com/display/XRAY/Import+Results

Have you already tried this REST path? Did you get an error?

HTH

Bruno Conde.

[sindhuri a]

Hi Bruno,

This link  http://xray.xpand-it.com/display/XRAY/Import+Results directs to page not found. Can you please provide example request to update existing test execution with cucumber json results.

Thanks

[skoli]

Hi Bruno, 

We have many Jira project and a seperate Jira project for Test case,

I wanted to create the test execution in a specific project.  As you mention in your response above that  we shall be able to specify the Project explicitly.

did you guys already release that api.. if yes, Can you please tell me what that json key is for specifying the  project key? 

Thanks!!

[Bruno Conde]

Hi @skoli

This depends on the endpoint you are using for importing execution results. Some endpoints provide directly the projectKey as a query parameter, others don't. We are aware of these inconsistencies and will address this in the future.

However, it is possible to import results and create Test Execution in any project you want just be using the multipart endpoint provided by Xray (for each report format). With this endpoint, it is possible to provide a JSON document that will be used to create the new Test Execution in Jira, as well as the report data of course.  Hence, you just need to specify the project key where you want to create the Test Execution.

https://confluence.xpand-it.com/display/XRAY/Import+Execution+Results+-+REST

Hope this helps,

Best Regards,

Bruno Conde.

[skoli]

Hi Bruno, 

thanks for the quick answer..  I am using  "/rest/raven/1.0/import/execution"  to create a new test execution. 

We have 2 project. 

1. Test case pool..  lets say TCP ( jira project key TCP)

2.  A product Project ( lets says ProjectKey CAL )

now my json file contains 

{info { testplanKey: CAL-123}

tests :[

{
"status": "PASS",
"testKey": "TCP-10"
},

]

}

in this case I expect the testexecution shall be created in Product Project ( i.e. CAL) 

but in reality it is getting created in TCP..

how to fix this?  Do yo have any advice?

[skoli]

shouldn't that be easy, if we use  the "ProjectKey" in the info part in json (same as testplanKey)  while using the /rest/raven/1.0/import/execution? 

or use the testplanKey to understand that test execution shall be created in the same JIRA project where testplan is created. 

what do you think ?  

[skoli]

Hi Bruno, in the mean time, I tried to use multipart api 

in that case I can not associate the test execution with a test plan. 

{
"fields": { "project": { "key": "TESP1"},
"summary": "Brand new Test execution created by multipart",
"issuetype": { "id": "11109" },
"customfield_11111" : [ "TESP1-19" ]
}

}

it say: 

customfield_11111: Field 'customfield_11111' cannot be set. It is not on the appropriate screen, or unknown.

Any help here?? 

[skoli]

Ok.. I am a bit further now..  fix the above error.. 

but now the date time format while creating the test execution.. giving me error

customfield_12426: Error parsing time: 2019-06-18T16:10:41+02:00
customfield_12427: Error parsing time: 2019-06-18T16:22:59+02:00

2.  Test results having also the date and time.. that are getting ignore/skip. ( no error)

"start": "2019-06-18T16:16:36+02:00",
"finish": "2019-06-18T16:16:41+02:00",
"status": "PASS",
"testKey": "TESP2-10"

So Could you please share what date/time format shall i passed in json to keep Xray rest API happy ?? 

[Tam Norris]

Hi, could you please explain. How you overcame your initial problem? I am also having issues with the multipart request. Im getting 500 internal application error.

[skoli]

i am not sure which initial problem you are reffereing to.. if you share your json data.. and may be i can spot the problem..  please reduce the test array to only one test..  

[Tam Norris]

Hi Sanju. Sorry I wasn't very clear. I meant the issue where you could not associate the test execution to the test plan. Have you been able to resolve that? I am sending the below request ...

I created the results file by following the instructions at "https://confluence.xpand-it.com/display/XRAYCLOUD/Testing+using+Cucumber+in+Java"

This json does not look exactly like the example provided in the docs for the cucumber multipart example, so this might be significant.

POST https://xray.cloud.xpand-it.com/api/v1/import/execution/cucumber/multipart HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: multipart/form-data; boundary="----=_Part_22_25189182.1562241869774"
MIME-Version: 1.0
x-access-token: xxxxxxxxxxxxxxIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnQiOiJiYjgwYmZjMS04NjIyLTNlOGQtYjg2OS00Yzk4Njc5MWExZDciLCJhY2NvdW50SWQiOiI1ZDEyMGQwMzFlMDYzYjBiY2Q3MGZjYjxxxxxx
Content-Length: 6897
Host: xray.cloud.xpand-it.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)

------=_Part_22_25189182.1562241869774
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: form-data; name="testPlanKey"

CDAF1234
------=_Part_22_25189182.1562241869774
Content-Type: text/javascript; charset=Cp1252; name=issueFields.json
Content-Transfer-Encoding: binary
Content-Disposition: form-data; name="info"; filename="issueFields.json"

{
"fields": {
"project": {
"id": "CDAFSBXC"
},
"summary": "Brand new Test execution",
"issuetype": {
"id": "10007"
},
"components" : [
{
"name":"Interface"
},
{
"name":"Core"
}
],
"customfield_10032" : [
"TES-38"
]
}
}
------=_Part_22_25189182.1562241869774
Content-Type: text/javascript; charset=Cp1252; name=results.json
Content-Transfer-Encoding: binary
Content-Disposition: form-data; name="results"; filename="results.json"

[
{
"line": 1,
"elements": [
{
"start_timestamp": "2019-07-04T12:03:21.893Z",
"line": 4,
"name": "a few cukes",
"description": "",
"id": "tam-test-plan-1;a-few-cukes",
"type": "scenario",
"keyword": "Scenario",
"steps": [
{
"result": {
"duration": 1041492,
"status": "passed"
},
"line": 5,
"name": "I have 42 cukes in my belly",
"match": {
"arguments": [
{
"val": "42",
"offset": 7
}
],
"location": "Stepdefs.I_have_cukes_in_my_belly(int)"
},
"keyword": "Given "
},
{
"result": {
"duration": 93013,
"status": "passed"
},
"line": 6,
"name": "I wait 1 hour",
"match": {
"arguments": [
{
"val": "1",
"offset": 7
}
],
"location": "Stepdefs.I_wait_hours(int)"
},
"keyword": "When "
},
{
"result": {
"duration": 19627,
"status": "passed"
},
"line": 7,
"name": "my belly should growl",
"match": {
"location": "Stepdefs.my_belly_should_growl()"
},
"keyword": "Then "
}
],
"tags": [
{
"name": "@TEST_CDAFSBXC-11"
},
{
"name": "@TESTSET_CDAFSBXC-7"
}
]
}
],
"name": "Tam Test Plan 1",
"description": "",
"id": "tam-test-plan-1",
"keyword": "Feature",
"uri": "classpath:io/cucumber/skeleton/Tam%20Test%20Plan%201.feature",
"tags": []
},
{
"line": 1,
"elements": [
{
"start_timestamp": "2019-07-04T12:03:21.899Z",
"line": 3,
"name": "a few cukes",
"description": "",
"id": "belly;a-few-cukes",
"type": "scenario",
"keyword": "Scenario",
"steps": [
{
"result": {
"duration": 93440,
"status": "passed"
},
"line": 4,
"name": "I have 42 cukes in my belly",
"match": {
"arguments": [
{
"val": "42",
"offset": 7
}
],
"location": "Stepdefs.I_have_cukes_in_my_belly(int)"
},
"keyword": "Given "
},
{
"result": {
"duration": 228693,
"status": "passed"
},
"line": 5,
"name": "I wait 1 hour",
"match": {
"arguments": [
{
"val": "1",
"offset": 7
}
],
"location": "Stepdefs.I_wait_hours(int)"
},
"keyword": "When "
},
{
"result": {
"duration": 27306,
"status": "passed"
},
"line": 6,
"name": "my belly should growl",
"match": {
"location": "Stepdefs.my_belly_should_growl()"
},
"keyword": "Then "
}
]
}
],
"name": "Belly",
"description": "",
"id": "belly",
"keyword": "Feature",
"uri": "classpath:io/cucumber/skeleton/belly.feature",
"tags": []
},
{
"line": 1,
"elements": [
{
"start_timestamp": "2019-07-04T12:03:21.900Z",
"line": 4,
"name": "a few cukes",
"description": "",
"id": "a-few-cukes;a-few-cukes",
"type": "scenario",
"keyword": "Scenario",
"steps": [
{
"result": {
"duration": 84480,
"status": "passed"
},
"line": 5,
"name": "I have 42 cukes in my belly",
"match": {
"arguments": [
{
"val": "42",
"offset": 7
}
],
"location": "Stepdefs.I_have_cukes_in_my_belly(int)"
},
"keyword": "Given "
},
{
"result": {
"duration": 1417386,
"status": "passed"
},
"line": 6,
"name": "I wait 1 hour",
"match": {
"arguments": [
{
"val": "1",
"offset": 7
}
],
"location": "Stepdefs.I_wait_hours(int)"
},
"keyword": "When "
},
{
"result": {
"duration": 36266,
"status": "passed"
},
"line": 7,
"name": "my belly should growl",
"match": {
"location": "Stepdefs.my_belly_should_growl()"
},
"keyword": "Then "
}
],
"tags": [
{
"name": "@TEST_CDAFSBXC-11"
}
]
}
],
"name": "a few cukes",
"description": "",
"id": "a-few-cukes",
"keyword": "Feature",
"uri": "classpath:io/cucumber/skeleton/tamFeature1.feature",
"tags": []
}
]
------=_Part_22_25189182.1562241869774--

[skoli]

Hi @Tam Norris 

sorry, i don not know anything about cucumber.. i was just using pure json format and using python to upload it to xray.

yes, I was able to solve the association of test plan (from another jira project e.g. TESP1-123)  with testcases ( that are in another jira project TESP2-12345), by using the split file. as pointed by @Bruno Conde 

-----issuefield

fields["project"] = {"key" : TESP1}                    

fields["issuetype"]         = {"id": "11109"}

fields["customfield_12436"] = ['TESP1-123']

-----

and test array 

[ { "testKey": TESP2-12345,

...

}

]

and key is manily finding the right customfield Id..  

[Tam Norris]

@skoliThanks for replying. So were you able to successfully associate a test execution to a test plan using the multipart api call? I am making some progress with the help of Xray support. I'm still getting errors but the errors are more logical errors than just the 500 Internal Application error. 

I am getting ...

{"error":"Error creating Test Execution - Field 'xrayFields' cannot be set. It is not on the appropriate screen, or unknown."}

The info I am sending in my info file is ..

{  
   "fields":{  
      "project":{  
         "key":"xxxxSBXC"
      },
      "summary":"Brand new Test execution 2",
      "issuetype":{  
         "name":"Test Execution"
      },
      "xrayFields":{  
         "testPlanKey":"xxxxSBXC-21"
      }
   }
}

JIRA support are helping me with this so if I am able to solve it with them I will post the solution here.

[Tam Norris]

The issue with the above json info file is that the xrayFields was at the wrong level of the json structure. It is not a child of fields. It should be as below.

{  
   "fields":{  
      "project":{  
         "key":"xxxxSBXC"
      },
      "summary":"Brand new Test execution 2",
      "issuetype":{  
         "name":"Test Execution"
      }
   },

"xrayFields":{  
         "testPlanKey":"xxxxSBXC-21"
      }
}

[Jamie Atkins]

Hello,

Yes that was the first thing I tried, I've tried it again though and am still seeing problems. I'm now sending to http://<URL>/rest/raven/1.0/import/execution

My request is now

"{"info" : {"summary" : "Test execution trial","description" : "Blah"},"tests" : [{"testKey" : "ET-880","comment" : "Successful execution","status" : "PASS"},{"testKey" : "ET-881","comment" : "Successful execution","status" : "PASS"},{"testKey" : "ET-883","comment" : "Failed execution","status" : "FAIL"}]}"

But the response I am receiving is (HTML tags removed)

HTTP Status 406 - Not Acceptable
Not Acceptable
The resource identified by this request is only capable of generating responses with characteristics not acceptable according to the request "accept" headers

As it appears that the issue is possibly with my "Accept" header I am also sending this

setRequestHeader "Accept", "application/json"

Please can you let me know if this is correct?

To be honest I'm new to Jira, XRAY, JSON and REST 

Thanks for your help

[Smriti Ajit]

Hey guys I am using curl with java. When I run the runner file it gives no error and my results is not uploaded on jira as well. PL help me out

[Jithin MJ]

Hi @Smriti Ajit  

I am relatevely new to this topic and would like to implement this result uploading using REST APIs.  Were you able to find a solution for your question (if so, could you please share the details?) also, are you specifying the code to result uploading within runner class?  Please help!!!

[durova]

Is it possible to integrate selenium webdriver tests (in BDD style) with XRAY and Bamboo? Or is it only for cucumber test?

[Luca Martinetto]

Hi Bruno, Jamie,

I'm using XRAY for JIRA and I'm trying to import a JSON result via Import interface on Test Execution issue.
I've got some questions:

1) How can I import a cucumber JSON format? I've browsed online help and so on and I found that I should be able to export the JSON formatting it properly. But it doesn't work. I have a lot of unhandled exceptions without any possibility to understand what's wrong (java.lang.exceptions)

2) If I convert the test automatic results to XRAY JSON format, I can successfully upload them but I cannot see the EXAMPLES results (I've raised a request about it: https://support.atlassian.com/servicedesk/customer/portal/23/JST-155207)

Thanks,
Luca 

[arunicsbit]

I am looking for the steps to Import Test cases to XRay Jira using REST API. I am looking for the JSON format I have to use to do the import. Can u help ?

[Jamie Atkins]

Ok it seems that removing the setRequestHeader is actually working now. Thank you for your help Bruno.

[Bruno Conde]

Great. I was just about to send you an example by using curl: curl -H "Content-Type: application/json" -X POST -u admin:admin --data @data.json http://yourserver/rest/raven/1.0/import/execution where the "data.json" is your JSON file.

[Smriti Ajit]

• I am using the same command in java but neither my file is uploaded nor I am getting any error on console. PL help