Checkstyle has errors but step passes

[Mike Lehan]

Running this

npm run --silent lint:tsc | npx @bartekbp/typescript-checkstyle > checkstyle.xml

On an example project produces a checkstyle file showing a TypeScript error. However because the program emits a 0 status code the step does not fail.

It would be a sensible assumption that if the report contains errors, it should fail the step even if the exit code is 0 (this would also permit multiple runs of different style checkers in a single step, emiting a report for each but not stopping if one encounters errors)

Is it possible to have the Pipe fail the step if the step is not yet failed and the report contains errors?

[Mike Lehan]

... (converted to comment - I'd delete it but there's no option to!) ...

[Patrik S]

Hello @Mike Lehan ,

I think in this case you could make use of grep and an "if" condition in your pipeline in order to check the report file for any string that indicates an error. If this error string is found, you can force the step to fail by manually returning an exit code different than zero.

Please allow me to share the example below :

image: atlassian/default-image:3

pipelines:
 default:
   - step:
     script:
       - echo "Success" > test_file.txt # creates a test_file containing "Success" string
       - if grep "error" "test_file.txt"; then exit 1; fi  # Check if the file contains the string "error". If yes, exit with 1, making the step to fail. 
       - echo "This is an error" > test_file.txt # creates a test_file containing "error" string
       - if grep "error" "test_file.txt"; then exit 1; fi # Check if the file contains the string "error". If yes, exit with 1, making the step to fail. 

In this example, the first "if" command will return zero exit code (success), because the test_file.txt does not contain the "error" string (it just had "Success" string). However, in the second "if" the test_file.txt now has the string "This is an error", which will match the grep "error" and thus execute the exit 1  command that will make that step fail.

You can adapt the code above to analyze the checkstyle file generated by the command you are executing, and look for a string your application outputs that indicates that an error has happened.

Hope that helps! Let me know in case you have any questions.

Thank you, @Mike Lehan .

Kind regards,

Patrik S

[Mike Lehan]

Thanks for your response. This seems to clash with the purpose of using this pipe. The pipe takes a checkstyle file and generates a report. That report has a status: passed/failed. If that report's a fail, the step, or at least the build, shouldn't be a pass.

This could be added as a config option in the event that would disturb existing usage, but I'd suggest that the principle of least surprise would lead a user to think that a failed report would mean a failed build.

So is it possible for the pipe itself to fail a build, rather than a user having to use something like grep (which is prone to its own errors, false positives etc)?

[Patrik S]

Hey @Mike Lehan ,

I'm afraid it's currently not possible to make the step fail when there's any failed test in the reports without using the workaround that I've mentioned before.

Having said that, I went ahead and created a feature request in our public issue tracker to implement the option to fail the step when any test report has a failed test: 

• https://jira.atlassian.com/browse/BCLOUD-21919

I would suggest you to add your vote there, since this helps both developers and product managers to understand the interest. Also, make sure you add yourself as a watcher in case you want to receive first-hand updates from that ticket. Please note that all features are implemented with this policy in mind: https://confluence.atlassian.com/support/implementation-of-new-features-policy-201294576.html

Thank you, @Mike Lehan !

Kind regards,

Patrik S