Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Jira XRay: Import JUnit results via REST API

Hi

I'm a newcomer to XRay.

I'm trying to import test results in JUnit format from DevOps to Jira XRay via REST API.

On the DevOps side I'm executing the following script on a Windows Server and receiving the error 404:

 

$pair = "name@domain.com:jira-token"
$bytes = [System.Text.Encoding]::ASCII.GetBytes($pair)
$base64 = [System.Convert]::ToBase64String($bytes)
$basicAuthValue = "Basic $base64"
$headers = @{ Authorization = $basicAuthValue }
$fileContent = [IO.File]::ReadAllText('C:\results\test-results.xml');
$fields = @{'appInfo'='{"name": "test","description": "test"}';'uploadFile'=$fileContent};
Invoke-RestMethod -Uri https://companyname.atlassian.net/api/v1/import/execution/junit?testExecKey=SQ-54 -Method Post -Headers $headers -ContentType 'multipart/form-data' -Body $fields;

 

I don't think that I have an authorization problem because the following command works:

 

$pair = "name@domain.com:jira-token"
$bytes = [System.Text.Encoding]::ASCII.GetBytes($pair)
$base64 = [System.Convert]::ToBase64String($bytes)
$basicAuthValue = "Basic $base64"
$headers = @{ Authorization = $basicAuthValue }
Invoke-RestMethod -Uri https://companyname.atlassian.net/rest/api/2/issue/SQ-54 -Method Get -Headers $headers 

What am I doing wrong with my import command?

1 answer

It seems that I have been trying to import the results to the wrong server: companyname.atlassian.net. 
For those who are using the XRay cloud the correct server to import the results to would be xray.cloud.xpand-it.com

Here is the Windows powershell script which allowed me to import the results to the Jira/XRay cloud:

$body = @{
"client_id"="****************************"
"client_secret"="********************************************"
} | ConvertTo-Json
$header = @{
"Content-Type"="application/json"
}
$token = Invoke-RestMethod -Uri "https://xray.cloud.xpand-it.com/api/v2/authenticate" -Method 'Post' -Body $body -ContentType "application/json"

Write-Host "Auth Result: $token"

curl.exe -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token" --data @C:\Temp2\test-results.xml https://xray.cloud.xpand-it.com/api/v2/import/execution/junit?testExecKey=SQ-75

Remark: The exe in curl.exe is important, without the exe not the real curl is executed but an alias of Invoke-RestMethod.

The file test.results.xml I used successfully to import the results had the following content:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite tests="15" failures="0" name="ut.com.xpandit.raven.service.impl.IssueDataSetTest" time="0.163" errors="0" skipped="0">
<testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="Test Story X" time="0.114"/>
<testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="Test Story Y" time="0.002"/>
</testsuite>

 

However, runing this script creates new test issues in Jira/XRay environment. I still don't understand how to update the status of an existing test instead of creating new tests while importing results via REST API. Any help would be appreciated.

Hi @cirwinch ,

importing results will autoprovision Test issues the first time; if you do import similar results, then existing Test issues will be reused as it will figure it out based on the Generic definition of the test that contains an unique identifier for the Test.

In sum, you don't need to worry about that unless that you want to report explicitly results for a Test that already exists using its issue key. For that you need to use a different format, such as TestNG. JUnit currently, doesn't yet support it (will support soon, using JUnit 5).

For an example, using TestNG, please see: https://docs.getxray.app/display/XRAYCLOUD/Testing+using+TestNG+in+Java 

Regards,

Sergio

Like cirwinch likes this

Hi Sergio
Thank you for your inputs. So the regular concept is to allow XRay to automatically handle tests by itself. Now I've managed to achieve what you explained with the following windows powershell script:

$body = @{
"client_id"="************************"
"client_secret"="*********************************************"
} | ConvertTo-Json
$header = @{
"Content-Type"="application/json"
}
$token = Invoke-RestMethod -Uri "https://xray.cloud.xpand-it.com/api/v2/authenticate" -Method 'Post' -Body $body -ContentType "application/json"

curl.exe -H "Content-Type: multipart/form-data" -X POST -F info=@testExecution.json -F results=@testResults.xml -F testInfo=@test.json -H "Authorization: Bearer $token" https://xray.cloud.xpand-it.com/api/v2/import/execution/junit/multipart


With this script I'm able to import JUnit results. I can see that XRay manages the import in such a way that for every test result it creates automatically a new test, if it does not already exist and adds all automatically created tests to one test execution issue.

The tests which are automatically created by XRay are not assigned to any Story(=Requirement in my case).

I tried to add properties with requirement IDs to the testcase section of the JUnit XML results file, as described by your JUnit documentation in the following link:

https://docs.getxray.app/display/XRAYCLOUD/Taking+advantage+of+JUnit+XML+reports  

And indeed after importing such a manually prepared JUnit XML result file, the tests were not only created by XRay but also assigned to the corresponding requirements (stories).

The question is how can I achieve the same thing automatically. I have no idea how to create a JUnit XML file which contains the requirement information which XRay requires to assign the tests to stories?

Hi @cirwinch , 

the only way to link to an existing story/requirement is to build a custom JUnit XML reporter using the JUnit 5 capabilities.

There is a PoC project here for JUnit5: https://github.com/Xray-App/xray-junit-extensions.

I created a temporary branch  for you: https://github.com/Xray-App/xray-junit-extensions/tree/legacy_support

 

It *should* work.

Regards,

Sérgio

Hi Sergio

Thank you for your comment.
I'm using cypress with mocha-junit-reporter to generate JUnit XML result files. And then I import the JUnit XML result files into XRay. I don't understand where the custom project you mentioned plays in here? I don't understand the connection between the project you mentioned (Java-Code) and cypress and mocha (Javascript-Code)? Could you give me some hints maybe?

Thanks and regards

@cirwinch , what I shared is a JUnit5 custom reporter.. in case you're using JUnit and Java platform. It seems that it isn't your case.

For cypress/javascript/mocha there is no out-of-the-box solution, AFAIK.

You would need to somehow customize the mocha junit reporter and somehow have a way to tag tests in cypress and have those tags getting mapped to the junit xml reported generated by mocha.. it's not straighforward I think but I don't know those frameworks extensively to provide you more insights.

Regards

Sérgio

I was able to get it working.

In your project you can use a custom mocha-junit-reporter. You can fork the one I have in my own personal

 

repo: https://github.com/bitcoder/mocha-junit-reporter

 

You then need to add it the your package.json

...
"mocha-junit-reporter": "https://github.com/bitcoder/mocha-junit-reporter",
...

 

You can create a test, and use a "requirement" to identify it as follows:

 

describe('My second test', function() {
it('Gets, types and asserts', { tags: 'something', requirement: 'CALC-123' }, function() {
cy.visit('https://example.cypress.io')

cy.contains('type').click()

// Should be on a new URL which includes '/commands/actions'
cy.url().should('include', '/commands/actions')

// Get an input, type into it and verify that the value has been updated
cy.get('.action-email')
.type('fake@email.com')
.should('have.value', 'fake@email.com') // error added on the test itself just to make it fail
})
})

 

This will produce a custom JUnit XML report that provides the linkage to the requirement, as describe at: https://docs.getxray.app/display/XRAYCLOUD/Taking+advantage+of+JUnit+XML+reports#TakingadvantageofJUnitXMLreports-LinkingTestswithRequirementsLinkingTestswithRequirements

@Sergio

Thank you very much. I've tested your code and it worked for me, but it seems that the parameter must be named 'requirement' as opposed to 'req'. Your current version generates a result file with the parameter 'req'. If I import it like that, XRay won't create the link between the generated test and the existing requirement. But if I manually modify 'req' to 'requirement' then it works.

Thank you and regards

Like Sergio Freire likes this

Great!

I fixed it meanwhile; thanks for the feedback.

Regards

Sérgio

Suggest an answer

Log in or Sign up to answer
TAGS

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you