Community
Q&A
Bitbucket
Questions
Bitbucket Pipelines marked as success even if one test failed

Bitbucket Pipelines marked as success even if one test failed

I have a test project that runs tSQLt tests on a separate service running the MSSQL 2017 server image.

The tests run correctly, and the output is in a correct XML format.

But even though one test failed:

The pipeline is still marked successful:

What am I missing here?

Thanks in advance for the help!

1 answer

1 accepted

Comments for this post are closed

Community moderators have prevented the ability to post new answers.

Post a new question

1 vote

Answer accepted

Hi,

A Pipeline is marked as success or failed based on the exit code of all the commands set. So if your command has the exit code 0, we will still mark that as successful. Most tools will return a non-zero exit code when a test fails. Perhaps you can configure your tool to have a different exit code.

The test output shown is read straight from the xUnit XML output. It doesn't influence the final result of the pipeline.

Thanks,

Phil

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Hi Phil,

Thanks for the answer! I am now able to have the Pipeline fail when a test fail. And if you do not mind, I have a follow-up question.

Currently my (simplified for the example) steps are:

# Run all tests
sqlcmd -b -Q "EXEC tSQLt.RunAll"
# Generate XML reports
sqlcmd -Q "EXEC tSQLt.XmlResultFormatter" -o test-reports/tsqlt.xml

Because I use the sqlcmd utility with the "-b" option, the first line returns an exit code different than 0.

But then I cannot execute the next steps (generate the reports). Do you have any idea how to solve this?

Regards

Andreas

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

As a workaround, I am using the following steps:

# Run all tests without the "_b" option, to not stop on errors
sqlcmd -Q "EXEC tSQLt.RunAll"

# Generate XML reports
sqlcmd -Q "EXEC tSQLt.XmlResultFormatter" -o test-reports/tsqlt.xml

# And finally, run all tests again, this time with the "-b" option, to make Pipelines fail if there is any error
sqlcmd -b -Q "EXEC tSQLt.RunAll

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Was this helpful?

Thanks!

Bitbucket

Forums

Q&A

Community resources

Support

Top groups

Community resources

Support

Learn

Community resources

Support

Events

Community resources

Support

Bitbucket Pipelines marked as success even if one test failed

1 answer

1 accepted

Comments for this post are closed

Was this helpful?

Thanks!

TAGS

Atlassian Community Events