It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Pipelines xUnit compatible test results

Paul Stegler Mar 02, 2018

What format(s) does bitbucket pipelines support for test results? I am using:

dotnet xunit -xml ./test-results/results.xml

 

and pipelines picks up the results.xml file but then when it parses it it detects 0 tests run even though the console output shows there were many tests run along with the results.

The documentation is very vague as to what they mean by xUnit compatible test results. Are they talking about the test results generated by xUnit.net? JUnit? Microsoft TRX?

 

The xml I'm getting back is

<?xml version="1.0" encoding="utf-8"?><assemblies timestamp="03/02/2018 12:02:08"> <assembly name="MyDomain.Tests.dll" environment="64-bit .NET Standard [collection-per-class, parallel (8 threads)]" test-framework="xUnit.net 2.3.1.3858" run-date="2018-03-02" run-time="12:02:08" total="17" passed="16" failed="1" skipped="0" time="0.193" errors="0"> <errors /> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+Classes" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+Classes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+Classes" method="Are_found" time="0.0290398" result="Pass" /> </collection> <collection total="5" passed="5" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" time="0.029"> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: -1, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0288192" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 6, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000509" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 0, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.000017" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 3, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000072" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 5, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000054" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" method="Are_found" time="0.0288412" result="Pass" /> </collection> <collection total="7" passed="7" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidatorTests" time="0.039"> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 0)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.030325" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 1)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000988" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 3)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000316" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 5)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.000021" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot; \&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0081974" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot;\&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.000223" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: null)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0001669" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" method="Are_found" time="0.0353826" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes.ReferencTypes_are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" method="ReferencTypes_are_found" time="0.0352717" result="Pass" /> </collection> <collection total="1" passed="0" failed="1" skipped="0" name="Test collection for MyDomain.Tests.Validation.FailMe" time="0.007"> <test name="MyDomain.Tests.Validation.FailMe.WillFail" type="MyDomain.Tests.Validation.FailMe" method="WillFail" time="0.0066377" result="Fail"> <failure exception-type="System.Exception"> <message><![CDATA[System.Exception : durp]]></message> <stack-trace><![CDATA[ at MyDomain.Tests.Validation.FailMe.WillFail() in C:\code\MyDomain\common\MyDomain.Tests\FailingTest.cs:line 15]]></stack-trace> </failure> </test> </collection> </assembly></assemblies>

 

2 answers

0 votes
prashant mvr Mar 27, 2018 • edited

when you say pipeline i assume that you are talking about bamboo build. 

Bamboo does not recognize xunit result output format thats why you see 0 results. 

One workaround is to convert the xunit results file output format to nUnit format which bamboo recognize's, so instead of xml in your command replace it with nunit and then give it a try.

dotnet xunit -nunit ./test-results/results.xml

Andrew Bowen Dec 16, 2018

The documentation states xUnit.

https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

But it doesn't? Excellent.

 

xunit.PNG

0 votes
Andrew Bowen Dec 16, 2018

JUnit format is supported.

 

I found this post, where they state xUnit format is not supported.

https://bitbucket.org/site/master/issues/15836/pipeline-xunitnet-test-results#comment-43922683

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

1,939 views 1 19
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you