Prevent Bamboo from failing job when failing tests are moved to quarantine

On Bamboo I have MSBuild job for building and running tests.

It works fine - but I want to use test quarantining option so when I have some test moved to quarantine (and this moved tests are only what fail) job will be marked as successful

I moved one failing test to quarantine but job is marked as failed


Job in Tasks I have Command named Test runner:

Executable = VSTest.Console

Argument = "WebServiceTestClient.dll" /Logger:trx

And after running job with failing test quarantined I get log

01-Apr-201410:59:44Total tests:62.Passed:61.Failed:1.Skipped:0.
01-Apr-201410:59:44TestRunFailed.01-Apr-201410:59:44Test execution time:10.1296Minutes
01-Apr-201410:59:44Failing task since return code of [C:\Program Files(x86)\Microsoft VisualStudio11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe "WebServiceTestClient.dll"/Logger:trx] was 1while expected 0
01-Apr-201410:59:44Finished task 'Test runner'
01-Apr-201410:59:44Starting task 'Parse test logs' of type 'com.atlassian.bamboo.plugin.dotnet:mstest'
01-Apr-201410:59:44Parsing test results...
01-Apr-201410:59:45Failing task since 1 failing test cases were found

Job is marked failed but then:

01-Apr-201410:59:45ChangingTaskResult to SUCCESS as all failed tests were quarantined.01-Apr-201410:59:45Finished task 'Parse test logs'01-Apr-201410:59:45Running post build plugin 'NCover Results Collector'01-Apr-201410:59:45Running post build plugin 'Clover Results Collector'01-Apr-201410:59:45Running post build plugin 'Artifact Copier'01-Apr-201410:59:45Finalising the build...

Unfortunately the output is:

enter image description here

This question was asked here hopefully somebody here give me right answer.

3 answers

We had this problem 2 weeks ago, I just had it run through a script then used the MSTest Parser instead. Yes, admittedingly it is a hack for functionality that should be builtin but it's not a show stopper anymore.

Would you mind providing more details about this script and configuration? We are admittedly a *nix shop but maintaining a Windows app during a migration process, so we defintely aren't windows scripting experts. How do you force the script to return 0?

The script always return 0 by default, you have to do extra work to make it fail a build. It just runs MSTest with testContainer, resultsFile, & category command line option. Then I use the MSTest Parser task to pickup the .trx file I just generated.

I'm going to need a little more detail. I tried doing this and it is definitely still returning an error code and it doesn't even make it to the MSTest Parser task. Here is the inline script I was using...

echo off
echo Running Tests writing results to testresults-%BAMBOO_BUILD_NUMBER%.trx
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" /testcontainer:./Source/UnitTest.DAL/bin/Debug/DSC.Dal.UnitTest.dll /resultsfile:testresults-%BAMBOO_BUILD_NUMBER%.trx

1 vote

A fixed plugin is available here:

You can install it on older Bamboo versions, but first, remove the old plugin from

There is newer version fixing additional problem when validadtion error occur. Not related to this but you may want to install the newest one.

It has to do with the fact that the MSTest Runner script is still returning non-zero result because there is a failing test. Bamboo sees this non-zero result from the script and assumes there is an issue. I'm surprised this doesn't work since Atlassian provides the MSTest Runner script. In my opinion this is a bug so I'm creating a support request with Atlassian.

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted Jan 15, 2019 in Statuspage

Introducing Statuspage Getting Started guides! First up: What is Statuspage?

Over the next several weeks we'll be sharing some of our Getting Started guides here in the community. Throughout this series of posts, we'd love to hear from customers and non-customers ab...

259 views 4 1
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you