Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Next challenges

Recent achievements

  • Global
  • Personal

Recognition

  • Give kudos
  • Received
  • Given

Leaderboard

  • Global

Trophy case

Kudos (beta program)

Kudos logo

You've been invited into the Kudos (beta program) private group. Chat with others in the program, or give feedback to Atlassian.

View group

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Help displaying custom test results

I have a custom build, that does not use a typical language that has Xunit tests, so I'm not using any particular testing setup. I do however have tests, I can format these in any format that is needed but I need to know what the format is.

I've tried the Xunit, JUnit, and another one that I'm not sure what it was now. In any case, at the end of my build the bitbucket pipeline finds the files, does _something_ to merge them but then finds no tests. I'm assuming that because it's not finding what it wants to find, it is not showing the "Reports" link or the tab for "Tests".

Since this is implicit behavior of the bitbucket pipelines, I have no insight into what might be going wrong here.

Is there some way to display more debugging information for this test merge process?

Is there a specific example of what test result format works for these?

 

Here is a snippet from the bottom of my log from my pipeline:

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Found matching test report file /opt/atlassian/pipelines/agent/build/test-results/all-test-results.xml
Found matching test report file /opt/atlassian/pipelines/agent/build/test-results/all-test-results2.xml
Found matching test report file /opt/atlassian/pipelines/agent/build/test-results/all-test-results3.xml
Finished scanning for test reports. Found 3 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

2 answers

1 accepted

0 votes
Answer accepted

Hi @Mark ,

Thank you for your question!

Note: the Tests tab will only show if you have failed tests to review.

Also, make sure that you provide the right path to tests reports (test-reports/report.xml).
Example for pytest:

script:
- pytest test/test.py --junitxml=test-reports/report.xml

More details you can find in the Test reporting in Pipelines guide.

So, I have a custom build. I'm not using a standard testing set up, but I am trying to figure out the format so I can format my test results so I can see them in the reports.

 

I've mocked up some results and put them in a /test-results directory. The system seems to see these as I see 

 

Finished scanning for test reports. Found 3 test report files.

 

However, it must not like what it sees because it always says:

 

Merged test suites, total number tests is 0, with 0 failures and 0 errors.

 

In my mocked test results I definitely have some "failures".

So, in the end it was the formats I was using. One of mine was close, but was apparently an older format of the junitxml like in your example.

I tried another results file with the format put out by pytest and it it was able to parse it. Thank you for your help!

Hi I am having similar issue. We are migrating of testing to bitbucket, we have been running it using robot framework. We are generating 3 files: 

Output: /opt/robotframework/reports/output.xml

Log: /opt/robotframework/reports/log.html

Report: /opt/robotframework/reports/report.html

 

But the piple is able to find only the xml file which I understand due to the format of the other files but it was not able to upload those files for me to view them, any idea why?

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4

Found matching test report file /opt/atlassian/pipelines/agent/build/sisko/reports/test-reports/output.xml

Finished scanning for test reports. Found 1 test report files.Merged test suites, total number tests is 0, with 0 failures and 0 errors.

 

Also the xml file only contains the data and the whole presentation is in those html files. Any idea how we can solve this issue?

I just generated some test results using pytest. I put those in the results directory to ensure they were in the right place, formatted correctly, and that they would display.

Once I had something that was working, I was able to compare what was working with what my tests were generating and I found differences. 

I think the test results are rather finicky on the exact formats. What I found that works is a format that looks like the following:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="none" name="somename.asm" skipped="0" tests="3" time="0" timestamp="2020-05-25T00:00:00.000000">
<testcase classname="SomeClass" file="somefile.asm" line="1" name="Some kind of name" time="0"></testcase>
<testcase classname="SomeClass" file="somefile.asm" line="2" name="Some kind of name" time="0"></testcase>
<testcase classname="SomeClass" file="somefile.asm" line="3" name="Some kind of name" time="0"></testcase>
</testsuite>
<testsuite errors="0" failures="1" hostname="none" name="someothername.asm" skipped="0" tests="3" time="0" timestamp="2020-05-25T00:00:00.000000">
<testcase classname="SomeOtherClass" file="someotherfile.asm" line="1" name="Some other kind of name" time="0"></testcase>
<testcase classname="SomeOtherClass" file="someotherfile.asm" line="2" name="Some other kind of name" time="0">
<failure message="This Failed">
...some details about failure...
...some details about failure...
...some details about failure...
...some details about failure...
...some details about failure...
</failure>
</testcase>
</testsuite>
</testsuites>

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket Pipelines

What We Learned When We Researched Open Source Vulnerabilities in 7 Popular Coding Languages

...hey are a part of us, shaping how we interact with the world around us. The same holds true for programming languages when we think about how different kinds of vulnerabilities raise their heads in t...

1,328 views 0 3
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you