Bamboo - NUnit Parser XML format

Brandon Stack March 25, 2019

What version of NUnit xml does the NUnit Parser require? I am able to run a custom script, that generates a TestResult.xml file, and the file is located. I can see in the file that most tests passed, none fail, and 2 are skipped. But in Bamboo, wherever the NUnit Parser reports the results, it claims every test has been skipped. My only explanation is that the XML format must not match. I am using NUnit Console v 3.10

3 answers

0 votes
Brandon Stack April 15, 2019

We did eventually figured this out. The answer was that we needed to use NUNit2 Format. We had tried formatting that way unsuccessfully and after going back over, a team member of mine noticed the example code had a bash control character in it (the ';', must have been meant for Windows). We enclosed it with double quotes:

--result="test-results/$x.xml;format=nunit2"

 

Now it works and picks up all of the tests. Thank you for the replies!

Brandon Stack April 15, 2019

To clarify my update, we are using NUnit.ConsoleRunner.3.9.0, and that needed to output the nunit2 format to work with our Bamboo agent's NUnit parser. 

Foong
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 15, 2019

What is the Bamboo version? NUnit 3 supposed to work in Bamboo 5.14+ BAM-12433

0 votes
Jimmy Seddon
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 15, 2019

@Brandon Stack I can't speak for the version of NUnit supported by the parser as we use that parser to read XUnit test results.  However, if you are getting results displayed it might not be the parsing of the results that is the issue, but the running of the NUnit tests on the Bamboo server.  Have you checked the build output while the unit tests are running to see if they are being run or skipped?

This is a snippet example of what I can see in my own logs:

build 02-Apr-2019 14:29:04      Common.Tests                     Total:  224, Errors: 0, Failed: 0, Skipped: 8, Time:   8.984s

And yes we legitimately have 8 tests that are being skipped at the moment.

I hope that helps.

-James

0 votes
Foong
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 15, 2019

Do you mean Bamboo shows more than 2 skipped?

Bamboo does not show passed test number. If it is only 2 skipped shown, what is the "Total tests" (there is a part that shows the total number of tests) shown in Bamboo? 

Brandon Stack April 15, 2019

 Atlassian Bamboo version 6.4.1 build 60405

Foong
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 16, 2019

Does the test case part have executed value?

 <test-case ...etc... executed="True">
Billy Milligan May 14, 2019

Hi @Foong ,

I'm not the original author, but I am experiencing the same "skipped test" behavior.  In my case, no, there is no "executed" value (see sample <test-case/>, below):

 <test-case name="AMPSExceptionConstructorTest()" fullname="AMPS.Client.Tests.AMPSExceptionTest.AMPSExceptionConstructorTest" methodname="AMPSExceptionConstructorTest" classname="AMPS.Client.Tests.AMPSExceptionTest" result="Passed" duration="0.01" asserts="0" />

 

In my case, I ran NUnit 3 from a linux job using using the the .Net Core 2.2 framework and using the following package references to test & produce NUnit results files:

 <ItemGroup>
<PackageReference Include="nunit" Version="3.10.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.10.0" />
<PackageReference Include="NunitXml.TestLogger" Version="2.1.36" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />

Failed tests show up properly, but any passing tests are instead marked as skipped.

FWIW, it appears that we are using Bamboo v6.2.3, if that factors into NUnit 2 vs 3 format support.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events