Best Practices for Clover integration with Bamboo?

[Dashboard]

I'm working to integrate clover with our existing bamboo build to do better reporting of the unit test results. As we all know, the clover build needs to be "separated" from the production build because of the instrumentation injected into the class files.

Does nay one have any "best practices" as to how that should be mapped into the Bamboo plan structure? E.g. Run a job for the test builds, then a separate job for the "production" build and deploy, or other setups to the "clover enabled plan".

[Marek Parfianowicz · Accepted solution]

Hi Brian,

Thank you very much for posting a great question. The answer is not so straightforward, however.

Everything depends on your needs, actually. It's worth to ask yourself few questions:

1) Do you treat code coverage as a quality gate in your production process? In such way that if, for instance, coverage drops down compared to a previous build or drops below certain threshold then the build should fail?

If yes, then it may be worth to create a Job with Clover in build plans for which such quality gate is required. The most probably such job should run before other ones (like building a production binaries), so you can create a stage for such job.

2) Do you need to have an immediate and constant feedback about code coverage? In such way, that you're willing to sacrifice build duration and wait longer to see build results since push to repository?

If yes, then you can enable Clover in existing plans (e.g. automatic Clover integration + with 'mvn test'). You will wait longer to see build results, but you'll have coverage data for each build.

If not, then you can clone a plan (or a job) and enable Clover in such clone. A benefit is that you'll have two builds running - a 'normal' one, which should finish faster and a 'cloverized' one which will take a little bit longer to complete.

3) How frequently do you need to check code coverage? After every commit / build? Daily? Weekly?

Depending on this, you can decide whether to integrate Clover in existing plans (or their clones) or to create one (or more) separate plans with Clover with their own build schedule. Please also note that it's possible to use Clover with Eclipse and IntelliJ IDEA - developers use Clover to have an immediate feedback about code areas covered by tests and which ones need more tests; as such feedback is practically continuous, they tend to check coverage data on a Bamboo server less frequently.

Cheers
Marek

[Marek Parfianowicz]

Few examples from our environment: Clover team has a dedicated "Clover with Clover" plan, which is scheduled to run weekly and which runs all available tests (unit, integration, functional) on all supported platforms. It takes few hours and measures overall coverage and generates an HTML report (with global and per-test coverage). Stash team has a "Stash Master with Clover" plan (which is a copy of a "Stash Master"), which is scheduled to run after every build of the "Stash Master". It runs core set of unit tests, so that feedback is quite fast. Bamboo team has two plans for "Functional Test Coverage" and "Unit Test Coverage", which run daily and runs unit and functional tests, but not for all supported platforms. As you can see, even within Atlassian, Clover usage varies a lot, depending on needs of the teams. One thing is common, however - we don't use code coverage as a quality gate, i.e. Clover is not configured to fail main build plans if coverage drops.

[Dashboard]

Thank you Marek!