Catkin tests aren't picked up by the scanner

[Kai Neubauer]

I am using catkin for my builds. The output is placed into "build/<pkg name>/test_results/<test_name>.xml"

These directories are not picked up by the scanner so I have to manually move them into a folder called TestResults. The command "catkin test" returns a non-zero value when tests fail and causes the pipeline to stop, meaning any commands after that point (like the move command) fail to be run. I have to force the pipeline to go to move them by running the following:

- catkin test || true
- mkdir TestResults
- mv build/**/test_results/**/*.xml TestResults

This means that the pipeline always succeeds even when tests fail. I don't want to parse the XML output to see if tests pass/fail. A workaround is running "catkin test" again after moving the XML files but this is a waste of build minutes so I'd rather not. I also found that the Tests tab is missing some of the failed tests. It says 8/19 tests failed but only two test outputs are visible for some reason. I can see all of the test output in the build tab but it's a pain to look through it to find the failed tests.

[Nick de Palézieux]

Being able to configure the test report directories would be great. Can't imagine that being a difficult thing to implement either.

What you can use in the meantime is adding the following as the last step in your pipeline. It prints out a test result summary and returns nonzero if a test failed.

- catkin_test_results build/

Note though that this will not indicate any tests that failed to build, which 

catkin test || true

glosses over.

So you might want to do something like

- if catkin test micro_bridge ; then catkin_test_ret=0; else catkin_test_ret=1; fi
- mkdir TestResults
- mv build/**/test_results/**/*.xml TestResults
- catkin_test_results build/ && [ $catkin_test_ret = 0 ]

[Norbert C]

Hi Kai,

Thank you for contacting Atlassian Community, my name is Norbert and I'm a Bitbucket Cloud Support Engineer, it's nice to meet with you! Welcome to the Atlassian Community!

I'd like to inform you that unfortunately currently we don't support this as the test report directories can't be configured, yet.

Currently we have a feature request for this as you can see in the link below:

* BCLOUD-15419 - Configure Searched Test Report Directories in the YAML

Our development team will give a first-hand update on that ticket if there's any progress made so I would suggest keeping a watch and vote for it.

Do note however that there's no ETA on enhancement request, and all enhancements are implemented with this policy in mind: Implementation of New Features Policy

Please let us know if you need any further assistance.

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support