Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Bamboo build succeeds even if test fails

Jonas Gucu March 6, 2023

Hello,

We have a build plan for a .NET 6 solution.

Our Bamboo version is 8.0.0 server edition.

This is our Tasks list:

image.png

 

Script task basically does "dotnet test" of all test projects and outputs results with the --logger parameter to a .trx file.

 

Problem is, even tho a test fails, this step just continues as successful. Additionally, the MSTest Parser final task parses tests as successful.

 

image.png

image.png

 

Why and how do I fix this?

2 answers

2 votes
Charlie Misonne
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 6, 2023

A Bamboo task will only fail if the exit code of the last command is 0. Microsoft's documentation says: "If all tests are successful, the test runner returns 0 as an exit code; otherwise if any test fails, it returns 1"

In you scenario it looks like the "test run" Bamboo task does return a 0 as exit code. Are you performing anything else after the dotnet test command that might return 0?

Jonas Gucu March 6, 2023

Okay, I think this error maybe not be what I think it is?

 

I left just the "dotnet test" step.

When "dotnet test" runs the project that has a COMPILATION error (see 2nd image above), it does not even run tests for the project, so no test result is given?

 

If a project has no compilation error, the test runs just fine and this is output in the log:

image.png

 

The project with the compilation error does not have this output.

 

Even with the compilation error in "dotnet test", the bamboo build successfully passes.

0 votes
Jonas Gucu March 8, 2023

Okay, so I redid our tasks list (not sure if first one is really needed):

 

image.png

 

Problems and changes:

1. Before we built 2 UI projects separately, that way if a test project had a compilation error, Bamboo could not see it. I swapped building two separate UI projects for the whole solution, now all compilation errors will be caught.

2. "dotnet-test" should fail the build if a test project has a compilation error, but it does not fail. I think this is a bug.

3. Swapped the order: before we used to test first in order not to waste time building, but now solution will be built first and tests ran later.

4. Artifact copy fails if "Clean working directory after each build" flag is checked in plan configuration.

image.png
So it seems that Bamboo copies artifacts AFTER cleaning working directory, which doesn't make sense? Should it not clean AFTER everything is done? Because in our case the build directory gets cleaned, artifacts disappear and artifact copy fails.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events