Clover history / threshold ratcheting seems inconsistent.


I have a project that I build with Gradle, using the ant tasks for Clover to check coverage. As part of the setup, I have configured history points and a threshold of 0 - i.e. coverage cannot decrease between builds.

The project is a multi-module project. The build fails when source code from 1 of the other modules changes - i.e. there is no change in coverage what so ever. Test & coverage is only configured for 1 of the modules.

here is how clover is configured to run (with gradle):

ant."clover-setup"(initString: "$buildDir/clover/test.db", tmpDir: $buildDir/clover/tmp") {
ant.fileset(dir: "src/main/java")
ant.fileset(dir: "src/test/java")

// move classes, compile instrumented source.

// Run tests

ant."clover-check"(threshold:"0%", historydir:"history")
ant."clover-report" { // report definition }

line 11 above checks the current coverage against history points. While testing this, I removed all history points except for one, made a change to a module that does not run tests, and re-ran the test with coverage. The coverage fails.

When coverage fails, new history points are not saved, i.e. the clover-check on a subsequent build gets re-run against the same checkpoint - and the summary shows completely different values for changes in coverage - really inconsistent. Any ideas as to why this is happening?

2 answers

1 accepted

This widget could not be displayed.

clover was in fact working correctly. Our in-container test cases (arquillian) reference data sources with varying data, and different conditions trigger different processing - so our application runs different code based on what data is returned.

we have set the threshold to a value that takes this into consideration, and are looking at updating test cases to wok with a predictable data set.

This widget could not be displayed.

Definitely seems to be a bug somewhere.

I ran the gradle task that invokes clover via ant now, it passes and generates a history point. I then commit 1 change to another module, that has no testing in it configured at all (the change I make is a wording change on a UI component, so absolutely no impact on coverage), and then re-run test with coverage enabled:

Total coverage of 44.26% did not meet last history point target of 44.28%
-22.2 to 60% com.package.web.goal.SomeGoal
-2.9 to 58% com.package.util.ChartingPeriodUtil

Clover is configured to fail the build if the threshold of 0 is not met. i.e. the failed coverage build DOES NOT record another history point.

Then, within my CI system, I re-run the task, without any change to source at all, and it passes, i.e. the threshold passes.

Any idea why this changes, or what info I need to provide in order to determine this?

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Wednesday in Teamwork

What teamwork quotes inspire you?

Hey everyone! My name is Natalie and I'm an editor of the Atlassian Blog and I've got a question for you: What's your favorite quote about teamwork?  We've compiled a list here, along with...

140 views 16 7
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you