How test reporting on Bitbucket Pipelines works?

Jack Nolddor _Sweet Bananas_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
July 15, 2019

Hi community,

I'm struggling to make test reporting work on Bitbucket Pipelines.
- My maven project defines a couple of JUnit tests.
- Test reports are generated by maven surefire plugin on the right place at first glance.


ksnip_20190715-181449.png

This is an excerpt of my pipelines log:

+ pwd
/opt/atlassian/pipelines/agent/build

+ ls -lia target/surefire-reports/*.xml
5522980 -rw-rw-rw-. 1 root root 62399 Jul 15 15:54 target/surefire-reports/TEST-ut.com.nolddor.capgras.rest.model.SwitchAuthorizationTest.xm


Which looks like good, as xml reports are located in one of the default scanned paths following the documentation:

Configuring test reporting

To enable test reporting, make sure that build test reports are generated in one of the supported default locations (with a directory depth of 3 levels):

./**/surefire-reports/**/*.xml

./**/failsafe-reports/**/*.xml

./**/test-results/**/*.xml

./**/test-reports/**/*.xml

 

The test report file scanner will begin searching from the base directory of your build: /opt/atlassian/pipelines/agent/build

Configuring test runners

Maven Surefire / Failsafe plugins

For Maven build jobs, no special configuration is required if you are using Maven Surefire Plugin or Maven Failsafe Plugin. The reports are automatically generated when the unit or integration maven test goals are executed.

4 answers

1 accepted

1 vote
Answer accepted
Daniel Santos
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 15, 2019

Hello @Jack Nolddor _Sweet Bananas_

Thank you for the detailed description. It was very helpful!

Once I read It I went right away to test it myself and observe the same behavior. Which is:

If the test passes 
    You will only see them reported in the left panel beside the pipeline step name.

If the test fails
    You will see a new tab with the reports for the tests that failed.


My first glance was to check our docs to see if this information was available there and it was:

Note: the Tests tab will only show if you have failed tests to review.

I guess I learned something today =]


Please confirm if this is the same scenario you see there otherwise we will need to investigate this a bit further.

Jack Nolddor _Sweet Bananas_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
July 15, 2019

Quite interesting and confusing behaviour hehehe
I expected to see Test tab whenever my build finished either fail or success.

Thanks for resolve this mistery.

Like # people like this
Daniel Santos
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 16, 2019

I guess it was intended by design assuming that passed tests will not be reviewed.
I'm glad to be helpful!

Tim Kovalev August 26, 2021

It still can be useful to review the test results to analyze which tests take most of the suite time.

3 votes
Taras Postument October 3, 2019

Hey guys,
here is the ticket Bitbucket team created for this issue: https://jira.atlassian.com/browse/BCLOUD-15716

The current issue status is "Gathering Interest", so please go and vote for it if you are still interested.

Thanks!

Jack Nolddor _Sweet Bananas_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
October 3, 2019

Thanks for sharing

1 vote
Adam Woloszyn May 19, 2020

I was scratching my head for a while as well - but same experience here. Successful tests do not show a tab, while failures do. Kinda stupid...but at least we know now :)

0 votes
Abes Dabir September 24, 2021

Thanks for the post. I got caught by this... for the 2nd time over the years perhaps. Upvoted the open issue.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events