Clover history / threshold ratcheting seems inconsistent.


I have a project that I build with Gradle, using the ant tasks for Clover to check coverage. As part of the setup, I have configured history points and a threshold of 0 - i.e. coverage cannot decrease between builds.

The project is a multi-module project. The build fails when source code from 1 of the other modules changes - i.e. there is no change in coverage what so ever. Test & coverage is only configured for 1 of the modules.

here is how clover is configured to run (with gradle):

ant."clover-setup"(initString: "$buildDir/clover/test.db", tmpDir: $buildDir/clover/tmp") {
ant.fileset(dir: "src/main/java")
ant.fileset(dir: "src/test/java")

// move classes, compile instrumented source.

// Run tests

ant."clover-check"(threshold:"0%", historydir:"history")
ant."clover-report" { // report definition }

line 11 above checks the current coverage against history points. While testing this, I removed all history points except for one, made a change to a module that does not run tests, and re-ran the test with coverage. The coverage fails.

When coverage fails, new history points are not saved, i.e. the clover-check on a subsequent build gets re-run against the same checkpoint - and the summary shows completely different values for changes in coverage - really inconsistent. Any ideas as to why this is happening?

2 answers

1 accepted

0 votes
Accepted answer

clover was in fact working correctly. Our in-container test cases (arquillian) reference data sources with varying data, and different conditions trigger different processing - so our application runs different code based on what data is returned.

we have set the threshold to a value that takes this into consideration, and are looking at updating test cases to wok with a predictable data set.

Definitely seems to be a bug somewhere.

I ran the gradle task that invokes clover via ant now, it passes and generates a history point. I then commit 1 change to another module, that has no testing in it configured at all (the change I make is a wording change on a UI component, so absolutely no impact on coverage), and then re-run test with coverage enabled:

Total coverage of 44.26% did not meet last history point target of 44.28%
-22.2 to 60% com.package.web.goal.SomeGoal
-2.9 to 58% com.package.util.ChartingPeriodUtil

Clover is configured to fail the build if the threshold of 0 is not met. i.e. the failed coverage build DOES NOT record another history point.

Then, within my CI system, I re-run the task, without any change to source at all, and it passes, i.e. the threshold passes.

Any idea why this changes, or what info I need to provide in order to determine this?

Suggest an answer

Log in or Sign up to answer
Community showcase

Scrum Roles Explained: the Do's and the Don'ts

Hello Community,  Today we are going to talk about the three Scrum Roles. There is the Development Team, the Scrum Master and the Product Owner. In my opinion these three are all really impo...

100 views 0 4
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you