How can I identify code coverage for individual JIRAs?

[Chris Kaknevicius]

If I have JIRA, Clover and perhaps Crucible and Fisheye, is it possible for me to somehow provide per-issue reports for code coverage of code committed against those JIRAs? I've seen it done in other systems, but can't seem to find a logical way to align code coverage to commits against specific JIRAs, nor can I figure out a way to list that data against issues in JIRA.

[Marek Parfianowicz · Accepted solution]

For me, the important thing is to determine how much of the new code committed as part of a specific JIRA has coverage. The most basic implementation might be just a number: 35% of the code added for this JIRA is covered by unit tests.

[...]

It sounds to me like there isn't such a plugin.

I'm not aware of any plugin capable of doing this. However, please consider a little bit simpler solution, which can produce quite reasonable code coverage results: instead of trying to measure a "jira-id-per-single-source-line" code coverage you could measure a "jira-id-per-source-file" coverage. This could be achieved as follows:

• instrument code with Clover, build and execute tests
• find all files which were modified in commits containing given JIRA ID (e.g. using git log)
• take this list of files and pass in <fileset includes=""> in the <clover-report> task

Thanks to this you can have XML report (with coverage statistics) for a given subset of files only.

Please note that you could run tests (and measure code coverage) once and next generate multiple different reports (each for different JIRA ID).

[Chris Kaknevicius]

Hi Marek,

That sounds like a reasonable solution, however in our case I'm not sure it would have the right impact. In a situation where an organization has nearly no coverage, a slight modification to a file would yeild very small coverage on that file unless they took steps to refactor and cover the entire class. In large classes this wouldn't be reasonable in most cases. It would however encourage refactoring and adding coverage to legacy code, but I think we need a bit of a baby steps approach at the moment.

Thank you

[Marek Parfianowicz]

Hi Chris,

If your company has lots of legacy code and nearly no coverage for it, then maybe just monitoring of global trends would be a sufficient solution? Please note that Clover allows to generate history snapshots and HTML/PDF historical reports.

Historical report can contain list of classes for which coverage has changed recently; it can also draw charts with LOC/% of coverage and other metrics.

See also:

• https://answers.atlassian.com/questions/164945/generate-reports-of-differences-with-clover
• https://answers.atlassian.com/questions/45213/how-can-clover-be-configured-via-maven-build-running-in-bamboo-to-instrument-and-report-on-only-changed-files
• Historical Report

Cheers
Marek

[Marek Parfianowicz]

Hello Chris,

Your requirement is indeed challenging :-)

First of all, in order to have code coverage you must run build and execute tests. Do you have any continuous integration system, like Bamboo?

Next, it's worth to define what does it mean a per-issue report. For example:

a) Is it a report for the whole application from test execution after the commit containing given JIRA issue ID? If yes, then it might be achieved by running a build after every commit (and having every commit isolated - Bamboo provides such feature). You could see build results on a 'Bamboo' tab in JIRA (on the issue details page).

b) Or is it even further isolation where a single line of code is matched against JIRA issue ID using information from SCM ("blame")? If yes, then there is no need to run separate build after each commit, i.e. few commits could be bundled into one build.

In such case you'd have to implement such integration from scratch, unfortunately. For instance:

• instrument code with Clover, compile and run tests, generate Clover XML report
• collect information about JIRA issue ID from SCM system for every source line (e.g. get list of files modified since last build, run blame to get line-revision mapping, run log to get JIRA IDs from comments)
• match "JIRA ID<->source line" information from SCM against "coverage<->source line" from Clover's XML report

See also:

https://answers.atlassian.com/questions/86543/clover-continuous-inspection

Cheers

Marek

[Chris Kaknevicius]

For me, the important thing is to determine how much of the new code committed as part of a specific JIRA has coverage. The most basic implementation might be just a number: 35% of the code added for this JIRA is covered by unit tests. This way, a team member can see high level information about coverage for issues in a sprint that might help them identify where work is needed.

It sounds to me like there isn't such a plugin. Even if I do follow the steps you've given, I suppose the best way to report it at the JIRA level might be to create a custom field that would be used to capture this information through REST updates?