Pipelines xUnit compatible test results

[Paul Stegler]

What format(s) does bitbucket pipelines support for test results? I am using:

dotnet xunit -xml ./test-results/results.xml

and pipelines picks up the results.xml file but then when it parses it it detects 0 tests run even though the console output shows there were many tests run along with the results.

The documentation is very vague as to what they mean by xUnit compatible test results. Are they talking about the test results generated by xUnit.net? JUnit? Microsoft TRX?

The xml I'm getting back is

<?xml version="1.0" encoding="utf-8"?><assemblies timestamp="03/02/2018 12:02:08"> <assembly name="MyDomain.Tests.dll" environment="64-bit .NET Standard [collection-per-class, parallel (8 threads)]" test-framework="xUnit.net 2.3.1.3858" run-date="2018-03-02" run-time="12:02:08" total="17" passed="16" failed="1" skipped="0" time="0.193" errors="0"> <errors /> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+Classes" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+Classes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+Classes" method="Are_found" time="0.0290398" result="Pass" /> </collection> <collection total="5" passed="5" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" time="0.029"> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: -1, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0288192" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 6, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000509" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 0, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.000017" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 3, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000072" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 5, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000054" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" method="Are_found" time="0.0288412" result="Pass" /> </collection> <collection total="7" passed="7" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidatorTests" time="0.039"> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 0)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.030325" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 1)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000988" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 3)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000316" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 5)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.000021" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot; \&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0081974" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot;\&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.000223" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: null)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0001669" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" method="Are_found" time="0.0353826" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes.ReferencTypes_are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" method="ReferencTypes_are_found" time="0.0352717" result="Pass" /> </collection> <collection total="1" passed="0" failed="1" skipped="0" name="Test collection for MyDomain.Tests.Validation.FailMe" time="0.007"> <test name="MyDomain.Tests.Validation.FailMe.WillFail" type="MyDomain.Tests.Validation.FailMe" method="WillFail" time="0.0066377" result="Fail"> <failure exception-type="System.Exception"> <message><![CDATA[System.Exception : durp]]></message> <stack-trace><![CDATA[ at MyDomain.Tests.Validation.FailMe.WillFail() in C:\code\MyDomain\common\MyDomain.Tests\FailingTest.cs:line 15]]></stack-trace> </failure> </test> </collection> </assembly></assemblies>

[Andrew Bowen]

JUnit format is supported.

I found this post, where they state xUnit format is not supported.

https://bitbucket.org/site/master/issues/15836/pipeline-xunitnet-test-results#comment-43922683

[prashant mvr]

when you say pipeline i assume that you are talking about bamboo build. 

Bamboo does not recognize xunit result output format thats why you see 0 results. 

One workaround is to convert the xunit results file output format to nUnit format which bamboo recognize's, so instead of xml in your command replace it with nunit and then give it a try.

dotnet xunit -nunit ./test-results/results.xml

[Andrew Bowen]

The documentation states xUnit.

https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

But it doesn't? Excellent.