Clover coverage metrics very slow

[Mathew Benson]

I just migrated from Jenkins to Bamboo 6.6.  My build and unit tests took about 20 minutes to complete on the previous Jenkins server.  It parsed the cobertura formatted xml file generated by gcovr, in under 30 seconds.  I added Bamboo tasks to:

1. Run gcov on each file
2. Generate a string containing every gcov file in the build.
3. Run gcov_to_clover.py with the generated string as input.

This works, but is painfully slow, taking hours to run the gcov_to_clover.py script with all those files.  This is a little frustrating since the previous solution I was using parsed the coverage files in seconds.  For those that are using Bamboo for C/C++ projects, is this our only option?

[Mathew Benson]

I'm not a strong python developer, so I'm not sure to optimize this.  But the bottleneck lies in line 405:

file_node.appendChild(line_node);

If I comment this line out, it obviously doesn't write all the correct data to the xml file, but execution time goes from hours to about 20 seconds.  

[Jeyanthan I]

Hi Mathew,

Did you try running the same in the respective Bamboo agent's command prompt (outside Bamboo)?

If you experience same slowness, it is likely that the issue is not on the Bamboo side.

[Mathew Benson]

Yes.  It is not on the bamboo side.  I didn’t expect that it was.  The appendChild function scales poorly as the container grows.  With a code base with hundreds of thousands of SLOC, that function gets called hundreds of thousands of times.  I abandoned trying to make that python script work and found another script created by somebody else.  This new script performs an XSLT transform from the cobertura XML output of gcovr to the clover XML format.  That script takes about 5 seconds to complete.  So I can collect coverage metrics in about 30 seconds again.

I knew the problem was in the python script.  It’s just that was the only recommended way to show c/c++ metrics that I could find here on the Atlassian boards.  Clover seems to work great on java projects using the maven and ant build systems.  I was frustrated thinking I might be the only one using Bamboo on c/c++ projects using cmake/ctest.

I did eventually get it to work, though I’m not really sure how clover integration benefits the project.  It only shows a single summary metric.  I ended up having to use gcovr to generate the detailed html reports anyway.  Independent of clover.  Bamboo has nice integration into the ctest test results, letting me easily submit jira tickets on new issues.  There doesn’t seem to be a way to apply a pass/fail coverage threshold, or submit tickets when the minimum coverage is not met.

[Jeyanthan I]

Sure. You may check for existing feature requests and create one if need be here: https://jira.atlassian.com/projects/CLOV/issues/