Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bitbucket Pipelines marked as success even if one test failed

I have a test project that runs tSQLt tests on a separate service running the MSSQL 2017 server image.

The tests run correctly, and the output is in a correct XML format.

But even though one test failed:
test_detected_and_failed.png

The pipeline is still marked successful:
pipelines_success.png

What am I missing here?

Thanks in advance for the help!

1 answer

1 accepted

1 vote
Answer accepted

Hi,

A Pipeline is marked as success or failed based on the exit code of all the commands set. So if your command has the exit code 0, we will still mark that as successful. Most tools will return a non-zero exit code when a test fails. Perhaps you can configure your tool to have a different exit code.

The test output shown is read straight from the xUnit XML output. It doesn't influence the final result of the pipeline.

Thanks,

Phil

Hi Phil,

Thanks for the answer! I am now able to have the Pipeline fail when a test fail. And if you do not mind, I have a follow-up question.

Currently my (simplified for the example) steps are:

# Run all tests
sqlcmd -b -Q "EXEC tSQLt.RunAll"
# Generate XML reports
sqlcmd -Q "EXEC tSQLt.XmlResultFormatter" -o test-reports/tsqlt.xml

Because I use the sqlcmd utility with the "-b" option, the first line returns an exit code different than 0.

But then I cannot execute the next steps (generate the reports). Do you have any idea how to solve this?

Regards

Andreas

As a workaround, I am using the following steps:

# Run all tests without the "_b" option, to not stop on errors
sqlcmd -Q "EXEC tSQLt.RunAll"

# Generate XML reports
sqlcmd -Q "EXEC tSQLt.XmlResultFormatter" -o test-reports/tsqlt.xml

# And finally, run all tests again, this time with the "-b" option, to make Pipelines fail if there is any error
sqlcmd -b -Q "EXEC tSQLt.RunAll

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

Powering DevOps with Bitbucket Server & Data Center

Hi everyone, The Cloud team recently announced 12 new DevOps features that help developers ship better code, faster   ! While we’re all excited about the new improvements to Bitbucket ...

1,869 views 0 7
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you