Bamboo - NUnit Parser XML format

[Brandon Stack]

What version of NUnit xml does the NUnit Parser require? I am able to run a custom script, that generates a TestResult.xml file, and the file is located. I can see in the file that most tests passed, none fail, and 2 are skipped. But in Bamboo, wherever the NUnit Parser reports the results, it claims every test has been skipped. My only explanation is that the XML format must not match. I am using NUnit Console v 3.10

[Brandon Stack]

We did eventually figured this out. The answer was that we needed to use NUNit2 Format. We had tried formatting that way unsuccessfully and after going back over, a team member of mine noticed the example code had a bash control character in it (the ';', must have been meant for Windows). We enclosed it with double quotes:

--result="test-results/$x.xml;format=nunit2"

Now it works and picks up all of the tests. Thank you for the replies!

[Brandon Stack]

To clarify my update, we are using NUnit.ConsoleRunner.3.9.0, and that needed to output the nunit2 format to work with our Bamboo agent's NUnit parser. 

[Foong]

What is the Bamboo version? NUnit 3 supposed to work in Bamboo 5.14+ BAM-12433

[Jimmy Seddon]

@Brandon Stack I can't speak for the version of NUnit supported by the parser as we use that parser to read XUnit test results.  However, if you are getting results displayed it might not be the parsing of the results that is the issue, but the running of the NUnit tests on the Bamboo server.  Have you checked the build output while the unit tests are running to see if they are being run or skipped?

This is a snippet example of what I can see in my own logs:

build 02-Apr-2019 14:29:04      Common.Tests                     Total:  224, Errors: 0, Failed: 0, Skipped: 8, Time:   8.984s

And yes we legitimately have 8 tests that are being skipped at the moment.

I hope that helps.

-James

[Foong]

Do you mean Bamboo shows more than 2 skipped?

Bamboo does not show passed test number. If it is only 2 skipped shown, what is the "Total tests" (there is a part that shows the total number of tests) shown in Bamboo? 

[Brandon Stack]

 Atlassian Bamboo version 6.4.1 build 60405

[Foong]

Does the test case part have executed value?

 <test-case ...etc... executed="True">

[Billy Milligan]

Hi @Foong ,

I'm not the original author, but I am experiencing the same "skipped test" behavior.  In my case, no, there is no "executed" value (see sample <test-case/>, below):

 <test-case name="AMPSExceptionConstructorTest()" fullname="AMPS.Client.Tests.AMPSExceptionTest.AMPSExceptionConstructorTest" methodname="AMPSExceptionConstructorTest" classname="AMPS.Client.Tests.AMPSExceptionTest" result="Passed" duration="0.01" asserts="0" />

In my case, I ran NUnit 3 from a linux job using using the the .Net Core 2.2 framework and using the following package references to test & produce NUnit results files:

 <ItemGroup>
<PackageReference Include="nunit" Version="3.10.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.10.0" />
<PackageReference Include="NunitXml.TestLogger" Version="2.1.36" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />

Failed tests show up properly, but any passing tests are instead marked as skipped.

FWIW, it appears that we are using Bamboo v6.2.3, if that factors into NUnit 2 vs 3 format support.