On Bamboo I have MSBuild job for building and running tests.
It works fine - but I want to use test quarantining option so when I have some test moved to quarantine (and this moved tests are only what fail) job will be marked as successful
I moved one failing test to quarantine but job is marked as failed
Job in Tasks I have Command named Test runner:
Executable = VSTest.Console
Argument = "WebServiceTestClient.dll" /Logger:trx
And after running job with failing test quarantined I get log
01-Apr-201410:59:44TestRunFailed.01-Apr-201410:59:44Test execution time:10.1296Minutes
01-Apr-201410:59:44Failing task since return code of [C:\Program Files(x86)\Microsoft VisualStudio11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe "WebServiceTestClient.dll"/Logger:trx] was 1while expected 0
01-Apr-201410:59:44Finished task 'Test runner'
01-Apr-201410:59:44Starting task 'Parse test logs' of type 'com.atlassian.bamboo.plugin.dotnet:mstest'
01-Apr-201410:59:44Parsing test results...
01-Apr-201410:59:45Failing task since 1 failing test cases were found
Job is marked failed but then:
01-Apr-201410:59:45ChangingTaskResult to SUCCESS as all failed tests were quarantined.01-Apr-201410:59:45Finished task 'Parse test logs'01-Apr-201410:59:45Running post build plugin 'NCover Results Collector'01-Apr-201410:59:45Running post build plugin 'Clover Results Collector'01-Apr-201410:59:45Running post build plugin 'Artifact Copier'01-Apr-201410:59:45Finalising the build...
Unfortunately the output is:
This question was asked here http://stackoverflow.com/questions/22803557/prevent-bamboo-from-failing-job-when-failing-tests-are-moved-to-quarantinebut hopefully somebody here give me right answer.
I'm going to need a little more detail. I tried doing this and it is definitely still returning an error code and it doesn't even make it to the MSTest Parser task. Here is the inline script I was using...
echo off echo Running Tests writing results to testresults-%BAMBOO_BUILD_NUMBER%.trx "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" /testcontainer:./Source/UnitTest.DAL/bin/Debug/DSC.Dal.UnitTest.dll /resultsfile:testresults-%BAMBOO_BUILD_NUMBER%.trx
A fixed plugin is available here:
You can install it on older Bamboo versions, but first, remove the old plugin from bundled-plugins.zip
There is newer version fixing additional problem when validadtion error occur. Not related to this but you may want to install the newest one.
It has to do with the fact that the MSTest Runner script is still returning non-zero result because there is a failing test. Bamboo sees this non-zero result from the script and assumes there is an issue. I'm surprised this doesn't work since Atlassian provides the MSTest Runner script. In my opinion this is a bug so I'm creating a support request with Atlassian.
Over the next several weeks we'll be sharing some of our Getting Started guides here in the community. Throughout this series of posts, we'd love to hear from customers and non-customers ab...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs