Importing test execution results with new test cases using Xray Json REST API

Shawn Hesch April 28, 2020

I am attempting to import a list of test case execution results to a JIRA project using the Xray REST API.


Using the Xray JSON format I have successfully uploaded execution results for existing tests. Unfortunately I have not been able to upload results for new tests. 


The JSON used for the upload attempt is: 


    "testExecutionKey" : "NXTP-40",

    "info" : {

        "summary" : "JSON Test Execution"


    "tests" : [


            "status" : "PASSED",

            "testInfoBean" :


                "summary" : "JSONCreatedTest",

                "projectKey" : "NXTP",

                "genericTestDefinition" : "tes.definition"






When this JSON is uploaded using the REST API or using the test execution importer tool in JIRA, an error is returned stating there are no valid tests

{"error":"There are no valid tests imported"}

Based on the JSON format outlined in the documentation I am unable to find which fields are incorrect or missing.


Any help pointing to how the JSON needs to be formatted to create a new test via test execution upload would be greatly appreciated.




6 answers

2 accepted

0 votes
Answer accepted
Rogério Paiva - Xray Xporter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
October 26, 2021


This feature has been released in Xray version 5.1.0.

Please check the documentation here:

Thank you.

Kind regards,
Rogério Paiva [Xray Support Team]

0 votes
Answer accepted
Shawn Hesch May 14, 2020

After discussing this issue with Xray support I was informed that isn't currently possible using the Xray JSON format and that one of the XML formats would have to be used to accomplish my goals.

Kartheeswaran Ramasamy
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
May 13, 2024

I am facing the same problem, I have framework net6.0 , and trying to generate Specflow test result in xml but I am failing in it, its not generating for me, could you give some idea how to create it

0 votes
rbalamu2 January 4, 2023

Hi Team,

Importing test execution results with new test cases/updating test cases using Xray CSV is possible? Kindly help. We are looking for a solution using csv.




0 votes
Michal Chmielewski October 12, 2021

Hi. Is there any information about when this issue will be fixed? The project I am working on relies heavily on the importing of test results, and we cannot move forward otherwise. It would be very helpful to be able to upload through XRay Json format. 

Thanx :) 

Eric Olson October 12, 2021

Hey Michal. I don't know why they did it this way, but they ended up forking this conversation into a new feature request on their jira board that they never linked here:

They just recently notified me that the feature was released in 5.1.0. I haven't had a chance to try it yet to confirm that it works (waiting on my Jira admins to update the plugin), but theoretically it's out there in the wild!

0 votes
Eric Olson August 14, 2020

This is a serious shortcoming, and it's very confusing from a design perspective. The functionality (creating new tests automatically) already exists with some of the XML formats (TestNG, for example), but it doesn't work with your own proprietary XRay JSON format.

The integration I'm building becomes orders of magnitude more complex as a result. Consider, instead of:

  1. Create execution payload
  2. For each test, create test run payload, using either known Jira key or some standardized signature
  3. Send to XRay API

It becomes:

  1. Create execution payload
  2. Query Jira API for my project, pulling down all issues with the appropriate issueType
  3. Iterate over all test runs, doing some custom matching between a signature of my choosing and whatever lives on the Jira tickets returned from the query.
    1. If a signature matches, modify the appropriate test run payload to now hold the testKey pulled from the Jira ticket
    2. If it doesn't match anything in the query, store it off to the side for the moment
  4. Now iterate over all of the unmatched runs, calling the Jira API to create a new issue (1 by 1).
  5. For each created ticket, modify the associated test run payload to have the new testKey
  6. Send to XRay API


Our company is large with a wide diversity of team processes. Some of our teams are fine manually tagging all of their tests - especially those who still have a mix of manual and automated testing, but a good chunk of our projects have shifted to a fully-automated paradigm with hundreds or thousands of tests, and having to retroactively apply thousands of pieces of metadata to their codebase will be untenable.

Please fix this, @José Domingues [Xray] and thanks for leading the way on this request, @Shawn Hesch 

Eric Olson August 14, 2020

Oh, as an addendum, this problem is kind of hinted at via this section of the doc:

It took me a really long time to find this section, however, and even when I did find it, it didn't say anything about the XRay Json format, so I kept struggling with it until eventually I searched and found this post. It probably should be made more visible, and it should enumerate which exact formats can use the feature and which can't, rather than just giving a couple examples of each.

0 votes
José Domingues [Xray]
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
May 14, 2020

Hi @Shawn Hesch ,

When importing execution results using the Xray JSON result format in a Continuous Integration environment, you can specify which Test Execution issue to import the results on using the testExecutionKey property. Alternatively, you can create a new Test Execution for the execution results and specify the Test Execution issue fields in the info object.

We suggest you check our documentation and adapt your report to the example you can see here:

If you have any more questions, please submit a support request in our Service Desk

Best Regards,

Team Xray

Shawn Hesch May 14, 2020

Thank you for the information @José Domingues [Xray],

In this case I was attempting to create a new test case, not test execution.

After discussing this issue with Xray support I was informed that isn't currently possible using the Xray JSON format and that one of the XML formats would have to be used to accomplish my goals.



Suggest an answer

Log in or Sign up to answer
AUG Leaders

Atlassian Community Events