Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Catkin tests aren't picked up by the scanner

Kai Neubauer
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 15, 2022

I am using catkin for my builds. The output is placed into "build/<pkg name>/test_results/<test_name>.xml"

These directories are not picked up by the scanner so I have to manually move them into a folder called TestResults. The command "catkin test" returns a non-zero value when tests fail and causes the pipeline to stop, meaning any commands after that point (like the move command) fail to be run. I have to force the pipeline to go to move them by running the following:

- catkin test || true
- mkdir TestResults
- mv build/**/test_results/**/*.xml TestResults
This means that the pipeline always succeeds even when tests fail. I don't want to parse the XML output to see if tests pass/fail. A workaround is running "catkin test" again after moving the XML files but this is a waste of build minutes so I'd rather not. I also found that the Tests tab is missing some of the failed tests. It says 8/19 tests failed but only two test outputs are visible for some reason. I can see all of the test output in the build tab but it's a pain to look through it to find the failed tests.

2 answers

1 vote
Nick de Palézieux
Contributor
October 20, 2023

Being able to configure the test report directories would be great. Can't imagine that being a difficult thing to implement either.

 

What you can use in the meantime is adding the following as the last step in your pipeline. It prints out a test result summary and returns nonzero if a test failed.

- catkin_test_results build/

Note though that this will not indicate any tests that failed to build, which 

catkin test || true

glosses over.

 

So you might want to do something like

- if catkin test micro_bridge ; then catkin_test_ret=0; else catkin_test_ret=1; fi
- mkdir TestResults
- mv build/**/test_results/**/*.xml TestResults
- catkin_test_results build/ && [ $catkin_test_ret = 0 ]
0 votes
Norbert Csupka
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 18, 2022

Hi Kai,

Thank you for contacting Atlassian Community, my name is Norbert and I'm a Bitbucket Cloud Support Engineer, it's nice to meet with you! Welcome to the Atlassian Community!

I'd like to inform you that unfortunately currently we don't support this as the test report directories can't be configured, yet.


Currently we have a feature request for this as you can see in the link below:

* BCLOUD-15419 - Configure Searched Test Report Directories in the YAML

 

Our development team will give a first-hand update on that ticket if there's any progress made so I would suggest keeping a watch and vote for it.

 

Do note however that there's no ETA on enhancement request, and all enhancements are implemented with this policy in mind: Implementation of New Features Policy

 

Please let us know if you need any further assistance.


Best Regards,
Norbert
Atlassian Bitbucket Cloud Support

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events