Where do I view pipeline test reports?

According to the current docs, maven test results are picked up automatically and test reports generated. I can see in the tear down section of my build pipeline that it is picking up the results (output below), but can't see the reports anywhere. Can someone please advise where to view test reports?

 

Finished scanning for test reports.

Found 22 test report files.

All test reports aggregated into 44 test cases

Uploading test results

Finished uploading test results.

2 answers

2 votes

@Jason Sheedy I have found that you can only view the results of failed tests. When you have a pipeline with failing tests, you will then be able to select a dropdown that appears where the Logs header is and select Test Results. Alternatively, you can click on the x number of failures message on the unit testing step and that will take you to the test results as well.

Hope this helps!

Thanks Andrew. It seems a like a bit of an oversight to not display passing tests. I can't think of any good reason why they wouldn't include it. Maybe that's only available in the Enterprise version. :/

I would also like to see reports where all tests passed.

We now display the successful tests too. But don't generate a full report. Is there anything specific you'd want to see in a larger report for a 100% tests passing?

Hi Philip. Could you tell me where to see passing tests? I see maven test results output in the logs, but nothing appears in the bitbucket user interface.

thanks

Hi @Jason Sheedy,

Were you previously seeing failed test results as well? Or have none shown?

If you have everything already set up, they should be showing on the step like so:

Screen Shot 2018-01-15 at 5.36.39 pm.png

If you don't have it set up, you should have debugging information about this in the 'Build Teardown' section of your pipeline. Note there was a bug that prevented them from displaying, which we have now resolved so you may only seem them on new pipelines.

Screen Shot 2018-01-15 at 5.40.10 pm.png

Which directory are your test outputs going to at the moment? Right now we're only reading from specific test report directory names listed here: https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

Thanks,

Phil

Thanks @Philip HodderLooks like my test reports are being captured in the build teardown, but I don't see anything in the UI to indicate failed or passing tests. 

Found 22 test report files.
All test reports aggregated into 44 test cases
Uploading test results
Finished uploading test results.

When you say "should be showing on the step", is that displayed in the log output of the individual build or the main pipelines screen that shows all builds?

Correction. I do see failing tests, but it's not displayed in the same way as the screenshot you posted. 

FailedBitbucketTest.png

@Jason Sheedy

Sorry, I just showed a subset of the page. Here's the whole result:

Screen Shot 2018-01-16 at 10.36.54 am.pngThe same view as you're looking at.

Thanks,

Phil

Thanks Phil. I'm not seeing the same thing. Could there be some difference because it's an academic team account? (See below)

Screenshot from 2018-01-16 12-45-08.png

@Jason Sheedy

Using an academic account is unlikely to be the issue here. It looks like you just have your log view expanded. Do you have an icon that looks like this on the top right of your screen?

Screen Shot 2018-01-16 at 1.49.59 pm.png

Larger screenshot here. Look at the top right (next to 'Download raw'):

Screen Shot 2018-01-16 at 1.50.15 pm.png

If so, when you click it you should see what I'm seeing.

If not, can you let me know what Browser + Version + OS you're using?

Thanks,

Phil

Oh wow. I feel kind of stupid. I see it now, but I've got to say that it's less than intuitive. Maybe a job for the UX team??

Thanks for your help.

No worries. I'll pass on the feedback. :)

Is there any permission needed to get setup before you can view test reports? base from the log it already created the report but I'm not sure why I wasn't able to access it or maybe it's just permission issue or I'm doing it the wrong way.

 

Screen Shot 2018-02-07 at 2.49.36 PM.png

@Philip Hodder

Hello,

I cannot see the test reports too. I'm on Windows 10, with both Chrome & Firefox (latest versions).

I tried with 1 or 2 steps, but still there is no dropdown next to "Logs" to switch to test reports. The tests should be visible since half of them fail. The screen is almost the same as for the previous post (except it is not the same project).

How can I view the test reports ?

Hi,

What do you see in the logs for the Build Teardown command? Are any tests being discovered?

Which directory are the tests being output to at the moment? We only check for reports in certain directories. Have a look at the documentation here to double check: https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

 

Thanks,

Phil

Hi,

I output the tests in the "test-results" directory, and they are found in teardown, but not shown even if half of them have failed... See the image below:

bp.png

Thanks,
TL

Ok, I think I understand: tests are shown only if the result code is not 0 (which is not the same as all tests succeed...)

Hi,

The Test Result view will be shown by default if you have failed tests + failed pipeline.

However, it looks like a bug that you can't navigate to the tests even though you have failures (but a green build). I'll open an internal ticket to fix this.

Thanks,

Phil

Hi @Philip Hodder

I can now see Test Result view but it doesn't show anything on my endScreen Shot 2018-02-14 at 2.25.39 PM.png

Looks like the XML isn't being parsed correctly. What build tool are you using? Are you able to show me the XML that is being output?

@Philip Hodder

I'm using codeception for automation test. Here's an example of the xml file result

 

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="acceptance (chrome-qa)" tests="13" assertions="83" failures="2" errors="0" time="203.458746">
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="loadLoginPage" class="WinePageCest" feature="load login page" assertions="0" time="9.928986"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="loginUser" class="WinePageCest" feature="login user" assertions="0" time="7.722183"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="redirectToMyCollectionsWines" class="WinePageCest" feature="redirect to my collections wines" assertions="0" time="7.060703"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyWinesPage" class="WinePageCest" feature="verify wines page" assertions="18" time="11.594282"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyAddWineModal" class="WinePageCest" feature="verify add wine modal" assertions="12" time="19.244257"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyCellar" class="WinePageCest" feature="verify cellar" assertions="1" time="8.617440"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyLocation" class="WinePageCest" feature="verify location" assertions="1" time="8.631113"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyBin" class="WinePageCest" feature="verify bin" assertions="1" time="8.732308"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyBrowseWinesDetailsCatalogs" class="WinePageCest" feature="verify browse wines details catalogs" assertions="24" time="35.186749">
<failure type="Codeception\Exception\ConditionalAssertionFailed">WinePageCest: Verify browse wines details catalogs
Failed asserting that on page /wines/browse/1789818
--&gt; Wines
Add Asset
WINES SUMMARYWINES COLLECTION131BROWSE WINESDRINK SOONLOCATIONS
2013 "Carlo V" il Rosso dell' Imperatore Merlot and Cabernet Venezie IGT
RED
2016 ”Podere Casa Guidi” di Giuseppe Leonelli Barbera Frizzante Emilia IGT
RED - SPARKLING
N.V. ”Podere Casa Guidi” di Giuseppe Leonell
[Content too long to display. See complete response in '/Users/lovie/Documents/Projects/web/tests/tests/_output/raw/' directory]
--&gt; contains "vintage1".

/Users/lovie/Documents/Projects/web/tests/tests/_support/_generated/AcceptanceTesterActions.php:858
/Users/lovie/Documents/Projects/web/tests/tests/_support/Page/WinePage.php:230
/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php:89
</failure>
</testcase>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="addWine" class="WinePageCest" feature="add wine" assertions="1" time="41.462510"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="browseWinesFromInventory" class="WinePageCest" feature="browse wines from inventory" assertions="1" time="16.380560"/>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="verifyBrowseWinesDetailsInventory" class="WinePageCest" feature="verify browse wines details inventory" assertions="24" time="23.722190">
<failure type="Codeception\Exception\ConditionalAssertionFailed">WinePageCest: Verify browse wines details inventory
Failed asserting that on page /wines/collection/1772564
--&gt; Wines
Add Asset
WINES SUMMARYWINES COLLECTION132BROWSE WINESDRINK SOONLOCATIONS
1876 ”Podere Casa Guidi” di Giuseppe Leonelli Vino Nouvo Emilia IGT
RED
1999 3 Bridges Sémillon Golden Mist
WHITE - SWEET/DESSERT
1999 Kevin Cabrera Grape Producer
RED
1999 Montecalvi Alta Valle della Greve IGT
RED

[Content too long to display. See complete response in '/Users/lovie/Documents/Projects/web/tests/tests/_output/raw/' directory]
--&gt; contains "vintage1".

/Users/lovie/Documents/Projects/web/tests/tests/_support/_generated/AcceptanceTesterActions.php:858
/Users/lovie/Documents/Projects/web/tests/tests/_support/Page/WinePage.php:230
/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php:116
</failure>
</testcase>
<testcase file="/Users/lovie/Documents/Projects/web/tests/tests/acceptance/Sanity/004-WinePageCest.php" name="logoutUser" class="WinePageCest" feature="logout user" assertions="0" time="5.175464"/>
</testsuite>
</testsuites>

Suggest an answer

Log in or Join to answer
Community showcase
Piotr Plewa
Published Dec 27, 2017 in Bitbucket

Recipe: Deploying AWS Lambda functions with Bitbucket Pipelines

Bitbucket Pipelines helps me manage and automate a number of serverless deployments to AWS Lambda and this is how I do it. I'm building Node.js Lambda functions using node-lambda&nbsp...

705 views 0 4
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot