Quarantined tests still causing build to fail in Bamboo
I am adding accessibility tests to a project and do not want their results to affect the build (ie failed tests won't fail the build). For context, I am using cypress-axe and have everything in one file named accessibility.spec.js
Quarantining them, if this feature works as I understand it, should work perfectly... but it doesn't. To test this I did as follows (step 0: build is currently passing):
- Write a new file with two deliberately failing tests and run the build. Bamboo catches the two failing tests and fails the build.
- Quarantine them and rerun. Quarantine ostensibly works insofar as recategorizing the tests into the "Quarantined/Skipped" category and displaying "Failed Tests" as 0. However, the build still fails.
The summary message is "No failed test found. A possible compilation error occurred." - Now skip the tests in their source code. That is, it('should... ') becomes xit('should... '). This is the odd part. With every test in that file skipped, Bamboo does more than skip them. It acts like the file doesn't exist and they aren't counted towards the total number of tests. The build passes.
Obviously these tests are still functioning as blockers and therefore the quarantine feature is failing to serve my use case, but this is my first week using Bamboo. I think it is likely my ignorance to blame. When the build fails, here's a screenshot of the logs. The two lines which stand out to me are the first and last:
tput: No value for $TERM and no -T specified
Failing task since return code of [scripts/build-for-deploy.sh] was 1 while expected 0
I can't really make heads of tails of the first, so any education there would be greatly appreciated. As for the second, it seems that the build-for-deploy script is still detecting the failed tests and failing that stage. Let me know if other information is needed. Thank you!
