Pipelines xUnit compatible test results

Paul Stegler March 2, 2018

What format(s) does bitbucket pipelines support for test results? I am using:

dotnet xunit -xml ./test-results/results.xml

 

and pipelines picks up the results.xml file but then when it parses it it detects 0 tests run even though the console output shows there were many tests run along with the results.

The documentation is very vague as to what they mean by xUnit compatible test results. Are they talking about the test results generated by xUnit.net? JUnit? Microsoft TRX?

 

The xml I'm getting back is

<?xml version="1.0" encoding="utf-8"?><assemblies timestamp="03/02/2018 12:02:08"> <assembly name="MyDomain.Tests.dll" environment="64-bit .NET Standard [collection-per-class, parallel (8 threads)]" test-framework="xUnit.net 2.3.1.3858" run-date="2018-03-02" run-time="12:02:08" total="17" passed="16" failed="1" skipped="0" time="0.193" errors="0"> <errors /> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+Classes" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+Classes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+Classes" method="Are_found" time="0.0290398" result="Pass" /> </collection> <collection total="5" passed="5" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" time="0.029"> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: -1, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0288192" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 6, expectedResult: False)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000509" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 0, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.000017" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 3, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000072" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange.Works_as_expected(value: 5, expectedResult: True)" type="MyDomain.Tests.Validation.ValidationFunctionTests+WithinRange" method="Works_as_expected" time="0.0000054" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" time="0.029"> <test name="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+OpenGenericClasses" method="Are_found" time="0.0288412" result="Pass" /> </collection> <collection total="7" passed="7" failed="0" skipped="0" name="Test collection for MyDomain.Tests.Validation.ValidatorTests" time="0.039"> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 0)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.030325" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 1)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000988" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 3)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.0000316" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_succeeds_as_expected(property: 5)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_succeeds_as_expected" time="0.000021" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot; \&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0081974" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: \&quot;\&quot;)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.000223" result="Pass" /> <test name="MyDomain.Tests.Validation.ValidatorTests.Validation_throws_as_expected(value: null)" type="MyDomain.Tests.Validation.ValidatorTests" method="Validation_throws_as_expected" time="0.0001669" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes.Are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ValueTypes" method="Are_found" time="0.0353826" result="Pass" /> </collection> <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" time="0.035"> <test name="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes.ReferencTypes_are_found" type="MyDomain.Tests.AssemblyExtentionsTests+ReferenceTypes" method="ReferencTypes_are_found" time="0.0352717" result="Pass" /> </collection> <collection total="1" passed="0" failed="1" skipped="0" name="Test collection for MyDomain.Tests.Validation.FailMe" time="0.007"> <test name="MyDomain.Tests.Validation.FailMe.WillFail" type="MyDomain.Tests.Validation.FailMe" method="WillFail" time="0.0066377" result="Fail"> <failure exception-type="System.Exception"> <message><![CDATA[System.Exception : durp]]></message> <stack-trace><![CDATA[ at MyDomain.Tests.Validation.FailMe.WillFail() in C:\code\MyDomain\common\MyDomain.Tests\FailingTest.cs:line 15]]></stack-trace> </failure> </test> </collection> </assembly></assemblies>

 

2 answers

0 votes
Andrew Bowen December 16, 2018

JUnit format is supported.

 

I found this post, where they state xUnit format is not supported.

https://bitbucket.org/site/master/issues/15836/pipeline-xunitnet-test-results#comment-43922683

0 votes
prashant mvr March 27, 2018

when you say pipeline i assume that you are talking about bamboo build. 

Bamboo does not recognize xunit result output format thats why you see 0 results. 

One workaround is to convert the xunit results file output format to nUnit format which bamboo recognize's, so instead of xml in your command replace it with nunit and then give it a try.

dotnet xunit -nunit ./test-results/results.xml

Andrew Bowen December 16, 2018

The documentation states xUnit.

https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

But it doesn't? Excellent.

 

xunit.PNG

Like makaron likes this

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events