[XRay] test execution import - map junit results to existing test issues

[andre.gans@dominolaser.com]

This question is about the junit import in the xray test management plugin.

After some automated tests I have a junit result file (from pytest). I'd like to upload the results to xray via its v2 api with python scripts. This works, but I have not yet managed to map specific test cases to their Jira issue counterparts.

When I upload the junit file, the test issues can be created automatically if they don't exist yet, sure. Subsequent uploads are mapped to the same tests but once e.g. the test names change they will be mapped to new test cases in Jira again. Therefore I think it's better to map the tests to their Jira issue key explicitly.

I tried different xml properties and attributes like "testKey" (seems to be used in json format), "test", "issue", "key" etc. in the testcase elements but they are not recognized by xray. The results will always be mapped to the test name.

I haven't tried the json format yet since it would require additional parsing, pytest does not seem to support the xray json format.

Is it possible to map the tests explicitly or is there a better approach?

[Sergio Freire - Xblend]

Hi andre.gans@dominolaser.com ,

unfortunately with JUnit XML "standard" format it's not possible to have those kind of properties.

I've implemented an open-source project where Java+JUnit users can use a JUnit enhanced XML report in order to support this. This handling is still not yet implemented on Xray side, but will be soon.

For other languages that generate JUnit XML reports the challenge is on having a way to customize the report generation.

As you're using pytest, there are some libraries available that you can try out.

One of them is: https://github.com/typhoon-hil/pytest-typhoon-xray

I think it generates a Xray JSON file. That format allows you to specify the test issue key, so that you can enforce results to be mapped against it.

Hope this helps, please let me know how it goes :) 

Regards,

Sergio

[andre.gans@dominolaser.com]

Thank you @Sergio Freire - Xblend for your quick reply! I will have a look at the library you suggested and will let you know how we'll proceed.

[C_ Bönitz]

Hello! I am trying to do the same here but I am using Cypress. I could not find yet a way to generate the XML reports in the Xray extended JUnit format (that can use the test_key property for test cases) from my E2E cypress tests so they can map directly to an existent Xray Jira Ticket. If I write manually the property for the test case in the .xml file, the mapping works perfectly and the test results get uploaded to the right test ticket in Jira. Do you have any suggestion @Sergio Freire - Xblend ? What do I need to achieve this step? Adjust/create a custom mocha reporter? We do use Maven but I this does not apply to cypress E2E tests, does it? I would be very very grateful for any help since I am searching for months after this and I'm trapped in a try/error loop... Thanks in advance.

[Raul Țifrea (External)]

@C_ Bönitz we're using junit-xray-cypress-reporter and within the spec files have it defined like so:

const jiraId = 'JIRA_001';

describe(

  'Test Title', { xray: { jiraID: `${jiraId}` } },

  () => {});

Then the junit.xml report will look like this:

<testsuite name="Test Title" timestamp="2024-01-24T10:53:48" tests="5" time="49.297" failures="0">

    <testcase name="Test Case Title" time="27.156" classname="Test Case Title">

      <properties>

        <property name="test_key" value="JIRA_001"/>

      </properties>

    </testcase>

[C_ Bönitz]

@Raul Țifrea (External) ,

I cannot thank you enough, this is perfect! Worked fine at the first try.

Best Regards

[Stefan Salzl]

Hi andre.gans@dominolaser.com ,

Is this still an issue or could you find a solution on this?

I ran into this problem with cypress tests yesterday and found out how test executions can be linked to existing tests.

Let me know if you are interested in some exchange.

Best
Stefan 

[andre.gans@dominolaser.com]

Hi @Stefan Salzl ,
we then proceeded with another test framework (for other reasons), but I'm still interested in your solution.

regards
Andre

[Stefan Salzl]

Hi andre.gans@dominolaser.com 

As mentioned I was experimenting with cypress. I read through a whole bunch of documentations on how/which format test result files need to look like. Ended up with a special reporter that creates JUnit xml. I guess there are better, more elegant and more efficient ways to do. Need to add that this is not made in an operating team/on scale for several 1000 tests. As I said: I am just experimenting as I am not a developer and just try to expand my knowledge and always seeking for new challenges :)

When I imported the result file the first time I ran into the same problem that new test case issues were create even though I wanted to cover/map to already existing test case issues.

When I opened the newly created (generic) test case. The result writes info from the xml to the definition of the test case in the form of "name classname.name" whereas

• classname = name of automated test (in my case the "it" part of the test)
• name= name of test suite (in my case the "describe" part of my test file) concat with classname (see in details below)

On my research through several documentations I read that an import could be mapped with this string added to existing test cases. So I did following:

• added this string to my existing/specific test cases
• deleted the previously (accidently) created test cases
• did the import of the result file again

And to my personal surprise and overwhelming happiness --> this was successfully mapped correctly :) :)

result.xml:

<?xml version="1.0" encoding="UTF-8"?>

<testsuites name="Mocha Tests" time="1.6059999999999999" tests="2" failures="0">

  <testsuite name="Root Suite" timestamp="2022-03-10T01:30:35" tests="0" file="cypress/integration/e2e/sauerteig_cypress_xray.spec.js" failures="0" time="0">

  </testsuite>

  <testsuite name="My First Test" timestamp="2022-03-10T01:30:35" tests="2" failures="0" time="1.6059999999999999">

    <testcase name="My First Test JT-612" time="0.71" classname="JT-612" failure="false" error="false" success="true">

    </testcase>

    <testcase name="My First Test JT-613" time="0.896" classname="JT-613" failure="false" error="false" success="true">

    </testcase>

  </testsuite>

</testsuites>

test script:

describe('My First Test', () => {

    it('JT-612', () => {

        ...some steps....

    })

    it('JT-613', () => {

        ...some more steps...

    })

})

Definition of generic test case (string to map):

Looking forward to your answer.

Best
Stefan

[andre.gans@dominolaser.com]

Thanks a lot for your insights! I went back to my playground test repository and found that some of the generic tests created via pytest actually had entries in the definition. However I can't remember whether I tried it that way.
Anyway, thanks for your information, I will try this when I'll play with pytest again.