XRay Import Execution Results Behave JSON tutorial problem

[gitcrashcourse]

Hi there! I'm trying to follow the Testing using Behave in Python tutorial. I can get the import to work, but the "The execution details displays the result of the Cucumber Scenario." does not work. 

Here's what I'm doing:

I'm creating a new Test Execution (say, PROJ-123).

I'm creating a new Automated[Cucumber] test (say, PROJ-234)

I'm creating a new Automated[Cucumber] test (say, PROJ-345)

I'm using the following feature file with Behave

@PROJ-123
Feature: Verify something

Scenario Outline: Verify something with <data>
Given I use the data <data>
Then the result is <result>


@PROJ-234
Examples:
| data | result |
| 1 | 1 |

@PROJ-345
Examples:
| data | result |
| 2 | 4 |

I'm running behave with:

behave -k --format=cucumber_json:PrettyCucumberJSONFormatter -o cucumber.json --junit --format=json -o reports/data.json x.feature

I'm importing the report with

curl -H "Content-Type: application/json" -X POST -u user:password --data @reports/data.json "https://jira.example.com/rest/raven/1.0/import/execution/behave"
{"testExecIssue":{"id":"574356","key":"PROJ-123","self":"https://jira.example.com/rest/api/2/issue/574356"},"testIssues":{"success":[{"id":"574408","key":"PROJ-234","self":"https://jira.example.com/rest/api/2/issue/574408"},{"id":"574409","key":"PROJ-345","self":"https://jira.example.com/rest/api/2/issue/574409"}]}}

But when I look at the Test Details for either PROG-234 or PROJ-345, it's empty:

I've also tried to import the Cucumber JSON test report:

curl -H "Content-Type: application/json" -X POST -u user:pass --data @cucumber.json https://jira.example.com/rest/raven/1.0/import/execution/cucumber
{"testExecIssue":{"id":"574356","key":"PROJ-123","self":"https://jira.example.com/rest/api/2/issue/574356"},"testIssues":{"success":[{"id":"574408","key":"PROJ-234","self":"https://jira.example.com/rest/api/2/issue/574408"},{"id":"574409","key":"PROJ-345","self":"https://jira.example.com/rest/api/2/issue/574409"}]}}

The result is exactly the same: empty Test Details for either PROG-234 or PROJ-345.

I'm using Jira Data Center v8.13.1 with Xray.

[gitcrashcourse]

@Rogério Paiva - Xray Xporter , I'm super confused now.

I did contact XpandIT and I was told that filling the execution details by importing the JSON reports isn't supported. You're now telling me that it is, in fact, supported, as long as the feature is first imported through the REST API.

So, what's the deal here, it is or it isn't supported?

[Rogério Paiva - Xray Xporter]

@gitcrashcourse , sorry there could be some communication problem or misunderstanding.

Let's do the following, please reply in the support ticket (that you have opened) and tell it is related to this community thread and that you have talked with me.

We have a lot of support tickets going on and I was unable to find yours.

If necessary we can schedule a call with you to clear all doubts.

Thanks.

[Rogério Paiva - Xray Xporter]

Hi @gitcrashcourse 

It is necessary to analyze the JSON report that is being sent to Xray.

Must likely the report is not correctly formated.

If it is a behave report it cannot be uploaded into a Cucumber Test case, it needs to be uploaded to a Generic Test case.

If you need further help, please contact the Xray Support (http://xraysupport.xpand-it.com) so that we can request additional information and investigate further.

Kind regards,
Rogério Paiva [Xray Support Team]

[gitcrashcourse]

Rogerio,

I should have read the tutorial more carefully. According to the tutorial:

{quote}

The test (specification) is initialy created in Jira as a Cucumber Test and afterwards, it is exported using the UI or the REST API. 

{quote}

I was expecting that importing the JSON reports would automatically fill that information, but it doesn't.

Regards,

[Rogério Paiva - Xray Xporter]

Hi @gitcrashcourse 

That statement is correct; however, it gives an overview of the whole process, and things a bit more complex than that.

When dealing with Cucumber test cases, it is necessary first to upload the "test (specification)," this means that you need first to upload the Cucumber feature file using this REST API:

https://docs.getxray.app/display/XRAY/Importing+Cucumber+Tests+-+REST

After the upload, it is necessary to export the feature file from Xray to have all the Jira keys mapping and use this exported feature file to run the automatic test.

This is the REST API to export the feature file from Xray:

https://docs.getxray.app/display/XRAY/Exporting+Cucumber+Tests+-+REST

In the end, the JSON report generated (that was based on the exported feature file) will have the correct mapping to import successfully into Xray.

I would like to suggest contacting the Xray Support (http://xraysupport.xpand-it.com) so that we can request additional information and help further.

Kind regards,
Rogério Paiva [Xray Support Team]