Issues/Bug With Importing Test Execution JSON Results with New Tests

Brian Grecho May 19, 2021

My Json input file looks like this for testing minimal, 

{
"info": {
"project" : "project"
},
"tests" : [
{
"status": "TODO",
"testInfoBean": {
"summary": "Automation Test",
"projectKey": "project",
"genericTestDefinition": "class-api.method-enablesomething"
}
}
]
}

What is expected, following this formatting guide and the bottom of this page, is that by using "genericTestDefinition" a new generic test should be created if a matching case is not found with an equal genericTestDefinition. 

"genericTestDefinition": "class-api.method-enablesomething"

What is observed is that, a manual test is created in our Jira/Xray instance. 

Additionally, when I try to use the manualTestDefinition field, I can not create a successful case with the required step field. I only receive back an error stating:  {"error":"Error instantiating bean. Field(s) tests -\u003e testInfoBean -\u003e manualTestDefinition do not follow the correct format.

where, again same reference, my JSON input is: 

{ 
  "info": {
    "project": "project" 
  }, 
  "tests": [ 
    { 
    "status": "TODO", 
    "testInfoBean": { 
        "summary": "CreatedTestManual", 
        "projectKey": "project", 
        "manualTestDefinition": [ 
            { "action": "step1" } 
         ] 
      } 
    } 
  ] 
}

1 answer

0 votes
Rogério Paiva - Xray Xporter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 21, 2021

Hi @Brian Grecho 

In Xray, each reporting format has its own REST API and Test Type associated.

For instance, Xray JSON format is to be used with Manual Test Types.

The information you found at the bottom of the page, is about the JUnit format that uses an XML reporting format and is of Generic Test Type.

So if you want to use Xray JSON, it will be a Manual Test Type, if you want to have a Generic Test Type, it is necessary to use one of the formats that work as Generic, like JUnit, NUnit, xUnit, TestNG, Robot.

About the second issue, your JSON is not following the correct format, please check the Xray JSON format here (https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results#ImportExecutionResults-XrayJSONformat).

If you need additional help, please contact the Xray Support (http://xraysupport.xpand-it.com ) so that we can request additional information and help further.

Kind regards,
Rogério Paiva [Xray Support Team]

Brian Grecho May 21, 2021

Thank you so much Rogerio for the reply!

ok so, i was using this same reference, 

starting at ID XrayNewTest:

testInfoBean: {
oneOf: [
{
$ref: "/TestCreateInfoManual"
},
{
$ref: "/TestCreateInfoCucumber"
},
{
$ref: "/TestCreateInfoGeneric"
}
]
},

I guess i am not understanding this formatting guide properly:

const TestCreateInfoManual = {
id: "/TestCreateInfoManual",
type: "object",
properties: {
summary: {
type: "string"
},
projectKey: {
type: "string"
},
manualTestDefinition: {
type: "array",
items: {
$ref: "/ManualStep"
}
},
requirementKey: {
type: "string"
},
labels: {
type: "array",
items: {
type: "string"
}
}
},
required: ["summary", "projectKey", "manualTestDefinition"],
additionalProperties: false
};

 and then ManualStep is:

const ManualStep = {
id: "/ManualStep",
type: "object",
properties: {
action: {
type: "string"
},
data: {
type: "string"
},
result: {
type: "string"
}
},
required: ["action"],
additionalProperties: false
};

 

and as posted above my formatting as i understood was:

{ 
  "info": {
    "project": "project" 
  }, 
  "tests": [ 
    { 
    "status": "TODO", 
    "testInfoBean": { 
        "summary": "CreatedTestManual", 
        "projectKey": "project", 
        "manualTestDefinition": [ 
            { "action": "step1" } 
         ] 
      } 
    } 
  ] 
}

 Can you help me see what I'm missing here?

thank you so much!

Suggest an answer

Log in or Sign up to answer