XRay Import Execution Results Behave JSON tutorial problem

gitcrashcourse March 18, 2021

Hi there! I'm trying to follow the Testing using Behave in Python tutorial. I can get the import to work, but the "The execution details displays the result of the Cucumber Scenario." does not work. 

Here's what I'm doing:

I'm creating a new Test Execution (say, PROJ-123).

I'm creating a new Automated[Cucumber] test (say, PROJ-234)

I'm creating a new Automated[Cucumber] test (say, PROJ-345)

 

I'm using the following feature file with Behave

@PROJ-123
Feature: Verify something

Scenario Outline: Verify something with <data>
Given I use the data <data>
Then the result is <result>


@PROJ-234
Examples:
| data | result |
| 1 | 1 |

@PROJ-345
Examples:
| data | result |
| 2 | 4 |

 

I'm running behave with:

behave -k --format=cucumber_json:PrettyCucumberJSONFormatter -o cucumber.json --junit --format=json -o reports/data.json x.feature

 

I'm importing the report with

 

curl -H "Content-Type: application/json" -X POST -u user:password --data @reports/data.json "https://jira.example.com/rest/raven/1.0/import/execution/behave"
{"testExecIssue":{"id":"574356","key":"PROJ-123","self":"https://jira.example.com/rest/api/2/issue/574356"},"testIssues":{"success":[{"id":"574408","key":"PROJ-234","self":"https://jira.example.com/rest/api/2/issue/574408"},{"id":"574409","key":"PROJ-345","self":"https://jira.example.com/rest/api/2/issue/574409"}]}}

 

But when I look at the Test Details for either PROG-234 or PROJ-345, it's empty:

 

Untitled.png

I've also tried to import the Cucumber JSON test report:

curl -H "Content-Type: application/json" -X POST -u user:pass --data @cucumber.json https://jira.example.com/rest/raven/1.0/import/execution/cucumber
{"testExecIssue":{"id":"574356","key":"PROJ-123","self":"https://jira.example.com/rest/api/2/issue/574356"},"testIssues":{"success":[{"id":"574408","key":"PROJ-234","self":"https://jira.example.com/rest/api/2/issue/574408"},{"id":"574409","key":"PROJ-345","self":"https://jira.example.com/rest/api/2/issue/574409"}]}}

The result is exactly the same: empty Test Details for either PROG-234 or PROJ-345.


I'm using Jira Data Center 
v8.13.1 with Xray.

3 answers

0 votes
Jesus David Alvarez Mazzo
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 23, 2023

I am having the same issue, was this solved?

0 votes
gitcrashcourse April 26, 2021

@Rogério Paiva - Xray Xporter , I'm super confused now.

 

I did contact XpandIT and I was told that filling the execution details by importing the JSON reports isn't supported. You're now telling me that it is, in fact, supported, as long as the feature is first imported through the REST API.

 

So, what's the deal here, it is or it isn't supported?

Rogério Paiva - Xray Xporter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 5, 2021

@gitcrashcourse , sorry there could be some communication problem or misunderstanding.

Let's do the following, please reply in the support ticket (that you have opened) and tell it is related to this community thread and that you have talked with me.

We have a lot of support tickets going on and I was unable to find yours.

If necessary we can schedule a call with you to clear all doubts.

Thanks.

Jesus David Alvarez Mazzo
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 24, 2023

I am having the same issue, was this solved?

0 votes
Rogério Paiva - Xray Xporter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 28, 2021

Hi @gitcrashcourse 

It is necessary to analyze the JSON report that is being sent to Xray.

Must likely the report is not correctly formated.

If it is a behave report it cannot be uploaded into a Cucumber Test case, it needs to be uploaded to a Generic Test case.

If you need further help, please contact the Xray Support (http://xraysupport.xpand-it.com) so that we can request additional information and investigate further.

Kind regards,
Rogério Paiva [Xray Support Team]

gitcrashcourse March 29, 2021

Rogerio,

I should have read the tutorial more carefully. According to the tutorial:

{quote}

The test (specification) is initialy created in Jira as a Cucumber Test and afterwards, it is exported using the UI or the REST API. 

{quote}

I was expecting that importing the JSON reports would automatically fill that information, but it doesn't.

Regards,

Rogério Paiva - Xray Xporter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
April 26, 2021

Hi @gitcrashcourse 

That statement is correct; however, it gives an overview of the whole process, and things a bit more complex than that.

When dealing with Cucumber test cases, it is necessary first to upload the "test (specification)," this means that you need first to upload the Cucumber feature file using this REST API:

https://docs.getxray.app/display/XRAY/Importing+Cucumber+Tests+-+REST

After the upload, it is necessary to export the feature file from Xray to have all the Jira keys mapping and use this exported feature file to run the automatic test.

This is the REST API to export the feature file from Xray:

https://docs.getxray.app/display/XRAY/Exporting+Cucumber+Tests+-+REST

In the end, the JSON report generated (that was based on the exported feature file) will have the correct mapping to import successfully into Xray.

I would like to suggest contacting the Xray Support (http://xraysupport.xpand-it.com) so that we can request additional information and help further.

Kind regards,
Rogério Paiva [Xray Support Team]

Jyothish PA
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
June 1, 2023

@Rogério Paiva - Xray Xporter  Could please share the documentation for the endpoint rest/raven/2.0/import/execution/cucumber/multipart. 

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
PRODUCT PLAN
FREE
PERMISSIONS LEVEL
Product Admin
TAGS
AUG Leaders

Atlassian Community Events