How to get code coverage with integration test against tomcat web application?

Robert Chan June 21, 2016

I have a maven project that has multiple modules which compiled as WAR files.

Then I have a separate test module that contains the integration test. This module would launch the web application via tomcat from the WAR files and then launch the tests.

So far I've modified the maven project to use cargo. The only coverage report that I have right now is only within the test module.

 

How to get code coverage of the code of the web application?

1 answer

1 accepted

1 vote
Answer accepted
Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 21, 2016

Hi Robert,

Did you have a chance to read this article: https://confluence.atlassian.com/display/CLOVER/Using+Clover+for+web+applications?

If not, then I encourage you to follow these steps.

 

Lack of code coverage for a web application can have few root causes:

  1. Application was not instrumented by Clover - e.g. ensure to run 'clover:setup' before 'compile'.
     
  2. WAR was not created with instrumented classes - e.g. despite code was instrumented, the packaging task took original classes (may happen with a forked build lifecycle via 'clover:instr' and artifacts containing classifiers); ensure that you see classes named like "<base class name>$__CLR<hash code>" in your WAR.
     
  3. Code coverage was not collected at runtime
     - clover.db was not found due to a different working directory and relative 'initstring' path used - see application log for "CLOVER:" errors
     - clover.db was found but flush policy was set to default and Tomcat was not shut down - change policy to 'interval' or 'threaded'

  4. Code coverage was collected but not copied for report generation - e.g. if 'initstring' is relative and your tests and application start from different working directories, then coverage recording files will be written in two locations; copy them to one location in such case

 

Cheers
Marek 

Robert Chan June 22, 2016

Ok, so I ran the following command:

mvn clean clover:setup install clover:aggregate clover:clover

What I have now is that in the modules are java files in <moduleA>/target/clover/src-instrumented
Also, I have <moduleA>/target/moduleA.jar

I extracted moduleA.jar and I noticed that there are 3 sets of file per class.
E.g. fileA.class, fileA$__CLR<hash code>.class, and  fileA$__CLR<hash code>$1.class
 

Does this mean I have done step #1 and #2 correctly?

In regards to #3:

I could veriried that tomcat is shut down properly via cargo before clover-maven-plugin is called.

Now I do run into this problem "clover.db was not found due to a different working directory and relative 'initstring' path used" as I am getting "[WARNING] No Clover database found, skipping report generation".

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 22, 2016

Yes. It seems that steps 1 & 2 are set up correctly.

 

Regarding step 3. In case you did not set value of <initstring> property for clover-maven-plugin, then it will use relative paths by default and will create clover.db file for each Maven module.

Check your logs for error messages like:

CLOVER: Unable to load the coverage database at <path> 
CLOVER: No coverage data will be gathered

 

In case your Tomcat has a different working directory then adding this flag:

-Dclover.initstring.basedir=/path/to/your/build/workspace

should point Clover runtime at correct location of databases. More details here: https://confluence.atlassian.com/display/CLOVER/JVM+properties. Alternative deployment scenarios are described here: https://confluence.atlassian.com/display/CLOVER/Using+Clover+in+various+environment+configurations

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 22, 2016

Now I do run into this problem "clover.db was not found due to a different working directory and relative 'initstring' path used" as I am getting "[WARNING] No Clover database found, skipping report generation".

Do you see coverage recording files in /target/clover directories for each module? They shall be named like "clover.db<some hash>" (you may also find "clover.db<some hash>.1" and "clover.db<some long hash>.s").

Do you see a merged Clover database? As you ran clover:aggregate it shall produce the cloverMerge.db file.

Can you confirm that you did not set singleCloverDatabase=true?

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 22, 2016

PS: feel free to create a ticket at support.atlassian.com - attaching entire build log with debug logging (mvn -X) may speed up problem resolution. 

Robert Chan June 22, 2016

Let me back up a bit:
At the project root directory I ran: 

mvn clean clover:setup install clover:aggregate clover:clover

Which created java files in root/<moduleA>/target/clover/src-instrumented
Also, I have root/<moduleA>/target/moduleA.jar

I extracted moduleA.jar and I noticed that there are 3 sets of file per class.
E.g. fileA.class, fileA$__CLR<hash code>.class, and  fileA$__CLR<hash code>$1.class

I also have root/<moduleA>/target/clover/clover.db created from that

Then the next step I did is go to the test module root/testModule/

 ran the following command:

mvn clean verify clover:aggregate clover:clover

Yes, I do see coverage recording files in each module.
This created additional clover.db files in root/<moduleA>/target/clover/
e.g. 3 of "clover.db<some hash>" and 3 of "clover.db<some hash>.1" 
In this particular case I have these 6 files along with clover.db  

The cloverMerge.db is only found from root/target/clover

I did not set  singleCloverDatabase=true anywhere

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 22, 2016

Could you explain why did you run "mvn clean clover:setup install clover:aggregate clover:clover" in root, followed by "mvn clean verify clover:aggregate clover:clover" in root/testModule? Does it mean that running "mvn install" goal in root did not compile the testModule? Does it mean that testModule is not listed as a sub-module of root in it's pom.xml?

Robert Chan June 22, 2016

Um, I am under the impression that clover:aggregate would able to merge coverage data generated by multi-module projects from the root.

Running mvn install in root does not compile the testModule.
Yes, the testModule is not listed as a sub module of the root in its pom.

Would this make a big difference? 

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 22, 2016

OK. It's clear now. Yes, it has an impact, because clover:aggregate merges only these databases, which belong to modules defined in the project. You can use the clover:merge task which looks for all clover.db files in given directory.

Please try the following: 

# instrument application
cd &lt;root&gt;
mvn clean clover:setup install
 
# instrument test code, deploy app to Tomcat, run tests   
cd &lt;testModule&gt;
mvn clean clover:setup verify
 
# go back to root to merge all databases generate a report
cd &lt;root&gt;
mvn clover:merge clover:clover
Robert Chan June 23, 2016

I got to the last command but am facing errors

mvn clover:merge clover:clover

Failed to execute goal com.atlassian.maven.plugins:clover-maven-plugin:4.1.1:merge (default-cli) on project spotbuy: The parameters 'baseDir' for goal com.atlassian.maven.plugins:clover-maven-plugin:4.1.1:merge are missing or invalid

then I ran 

mvn clover:merge clover:clover -Dmaven.clover.merge.basedir=&lt;root&gt;

You must specify one or more coverage databases to merge using a nested  <cloverdb> or <cloverdbset> element

 I see that for ANT there's an example:

&lt;clover-merge initString="mergedcoverage.db"&gt;
    &lt;cloverDb initString="projectAcoverage.db"/&gt;
    &lt;cloverDb initString="projectBcoverage.db" span="30 mins"/&gt;
 &lt;/clover-merge&gt;

How do I do the same in Maven or via command line?

Robert Chan June 23, 2016

I managed to get the db merged by running

mvn clover:aggregate clover:clover instead.

I do have a question in regards to this.

In the log I see that:

Merging database 1 of 19: &lt;root&gt;/module1/target/clover/clover.db
...
Merging database 19 of 19: &lt;root&gt;/module19/target/clover/clover.db

in this case, in <root>/module1/target/clover/ there are other clover.db<hashcode> files created after running:

mvn clean clover:setup verify

does that mean that these clover.db<hashcode> files gets incorporated when the db merge is happening?
I just want to understand the relationship between the clover.db vs clover.db<hashcode>  

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 23, 2016

The message:

You must specify one or more coverage databases to merge using a nested <cloverdb> or <cloverdbset> element.

is printed by the Ant <clover-merge> task. It's because the Maven clover:merge goal executes the the Ant task under the hood.

This message means that it has no databases on the list to be merged. It means that the Maven clover:merge task did not found any database files in a folder specified by you.

It searches for files in the ${maven.clover.merge.basedir} directory matching pattern

${maven.clover.merge.includes}. The default value is "*.db". Try changing it to "**/*.db".

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 23, 2016

Does that mean that these clover.db<hashcode> files gets incorporated when the db merge is happening?

Yes. Exactly. 

I just want to understand the relationship between the clover.db vs clover.db<hashcode> 

The clover.db is a database, which stores information about code structure (classes/methods/statements etc). The clover.db<hashcode> are coverage recording files, which contain information how many times given method or statement was executed. The first one is created during compilation, the latter are created at runtime.

Falak Shah August 1, 2016

Can I use clover for following purpose:

--> Development system code uses ant to generate the war file. Tomcat is running using gendered war file.

--> Test automation code is different project and use testNG and maven to start the automation execution

-->After the automation code execution I need the code coverage from the war file.

--> Both development code set up and automation code are on same machine

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 1, 2016

Hi Falak, yes of course, you can setup Clover in such configuration.

Do you want to collect code coverage for your application code only?

Or maybe also for automation code?

Do you want to track per-test coverage, i.e. which testNG test case covered which parts of application?

Or it's enough for you to see just global coverage, i.e. that given source line was executed?

Falak Shah August 2, 2016

We need code coverage of application code only with per test coverage.

But right now I am executing same test case with different data in parallel. so in this case per test coverage is possible or not? 

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
August 2, 2016

In order to get per-test coverage you have to:

  • instrument both application and test code
  • configure 'distributed per-test coverage' feature (because your application and tests are running in separate JVMs)
  • run tests sequentially, not in parallel (because the per-test coverage feature is time-based and not thread-based)
kvallish October 29, 2019

Hi, 

I was able to get the per-test coverage and I have verified everything, but there is just one problem. Am running a testNG suite having 8-10 classes but in the report I get to see far fewer (1 or 2) per-test coverage reports. Have described my problem in detail at https://community.atlassian.com/t5/Clover-questions/Per-test-coverage-does-not-report-all-test-cases/qaq-p/1208868 .

 

Please help.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events