Generate clover report from 2 test runs' recording data

Hi,

 

I'm trying to generate combine report from 2 different testbeds on same packages: unit test and system test. What I found is the combined report is exactly the same as my unit test result, same coverage number, same hit number on lines.

 

Here is my setup to get the combine report:

  1. run unit test using Maven, get clover.db and clover.db* record data.
  2. Run system test with the instrumented binary. Got clover.db and clover.db* record data.
  3. Copy over clover.db* record data into the same directory of step2.
  4. Run command line command like this:
    sudo java com.atlassian.clover.reporters.html.HTMLReporter -i $STEP2_DB_DIR/clover.db -o /output_dir

What is wrong in my setup?

 

Thanks,

Will

 

2 answers

By turning the debug info, I found following log while generating report, could this be the reason? If so, how can we fix it?

 

It might because the builds I used in those 2 testbed has different directory?

 

Read data for file "/tmp/clover/ccp/clover.dbs124v8_ityee58c.1": FileBasedCoverageRecordingTranscript[header=Header[dbVersion=1475751270364, writeTimeStamp=1475761955725, format=0], coverageSum=80111, hitCounts.length=34485]
Global recording file:
file=RecordingTranscripts.FileRef[datafile=/tmp/clover/ccp/clover.dbs124v8_ityee58c, testRecording=false, typedTestId=-1, runId=0, hash=1694832164, timestamp=1475761666044]
dbVersion=1475751270364 (Thu Oct 06 03:54:30 PDT 2016)
writeTimeStamp=1475761955725 (Thu Oct 06 06:52:35 PDT 2016)
Ignoring coverage recording FileBasedCoverageRecordingTranscript[header=Header[dbVersion=1475751270364, writeTimeStamp=1475761955725, format=0], coverageSum=80111, hitCounts.length=34485] because no FileInfo supports its coverage range

Hi Marek,

When I merge 2 application's database into one, I hit this problem again.

Steps I did:

  1. Compile application A and test it, get clover.db and clover.db* record files
  2. Compile application B and test it, get clover.db and clover.db* record files
  3. Merge A and B's clover.db and put it into the report dir
  4. Copy A's clover.db* record files into report dir
  5. Copy B's clover.db* record files into report dir
  6. Run html generate command to generate report:
    1. java com.atlassian.clover.reporters.html.HtmlReporter -i clover.db -o clover_html

When I generate separate for each of the applications, I was able to see 65% percentage coverage for one application and 55% for another application. However, the combined report shows 15% only. 

Also, the covered element is even smaller than one application's report.

What could be the issue?

Thanks,

Will

 

In case you merge two or more clover.db databases, please ensure that all of them contain distinct sets of classes. In other words, if the same class is repeated in two (or more) databases, then Clover will pick coverage for this class from only one database.

 

Such situation can happen when you perform compilation of the same source code twice, for instance:

unit testing:

mvn clean clover:setup test -> will produce one clover.db

integration testing:

mvn clean clover:setup verify -> will produce another clover.db, but the same application code was compiled

 

You wrote that:

Run system test with the instrumented binary. Got clover.db and clover.db* record data

"got clover.db" suggests that you compiled the code. Am I correct?

You wrote that:

Merge A and B's clover.db and put it into the report dir

Copy A's clover.db* record files into report dir

Copy B's clover.db* record files into report dir

How did you merge A and B clover.db files? Did you use CloverMerge command for this purpose (or alternatively the clover:merge Maven goal or the <clover-merge> Ant task)?

I'm asking, because in case you used CloverMerge, then there is no need to copy recording files - coverage data is already merged in cloverMerge.db.

You wrote that:

java com.atlassian.clover.reporters.html.HtmlReporter -i clover.db -o clover_html

Is the "clover.db" the merged database? I'm asking because it's typically named "cloverMerge.db".

In case your build setup requires compiling the same application code twice, then setup similar to the following could work:

  1. run unit test:
       compile code and run unit tests with Clover
       clover.db and recording files will be created
  2. run system test:
       copy clover.db created in step 1 to this build
       compile code and run system tests with Clover
       clover.db will be updated and recording files will be created
  3. generate Clover report
       take updated clover.db from step 2,
       take recording files from step 1 and step 2 (put in the same directory as clover.db)
       run HtmlReporter
       note that there's no need to call CloverMerge task


The above is a general hint. I could tell you more if I could analyse details of your build setup. Feel free to create a ticket at support.atlassian.com. 

    Hi Marek,
      Please check my inline reply. Thanks for your prompt reply!



    "got clover.db" suggests that you compiled the code. Am I correct?
      Will Zhang: Yes, that's correct. We use maven clover plugin in this case to compile the code. And we copy over the generate clover.db into test machine to run tests and collect coverage data.
  1. How did you merge A and B clover.db files? Did you use CloverMerge command for this purpose (or alternatively the clover:merge Maven goal or the <clover-merge> Ant task)?

    I'm asking, because in case you used CloverMerge, then there is no need to copy recording files - coverage data is already merged in cloverMerge.db.

    You wrote that:

    java com.atlassian.clover.reporters.html.HtmlReporter -i clover.db -o clover_html

    Is the "clover.db" the merged database? I'm asking because it's typically named "cloverMerge.db".

    Will Zhang: We merged the database using CloverMerge command. And I just tried not copying over record file after merge, it still shows less coverage number, as 15.6%

 

  1. Marek Parfianowicz [Atlassian]

    In case your build setup requires compiling the same application code twice, then setup similar to the following could work:

    1. run unit test:
         compile code and run unit tests with Clover
         clover.db and recording files will be created
    2. run system test:
         copy clover.db created in step 1 to this build
         compile code and run system tests with Clover
         clover.db will be updated and recording files will be created
    3. generate Clover report
         take updated clover.db from step 2, 
         take recording files from step 1 and step 2 (put in the same directory as clover.db)
         run HtmlReporter
         note that there's no need to call CloverMerge task


    The above is a general hint. I could tell you more if I could analyse details of your build setup. Feel free to create a ticket at support.atlassian.com

    Will Zhang: I think I'm almost exactly the same setup as you mentioned above and able to get a merged report of my unit test report and system test.

    However, the problem now is that I'm having 2 different applications that has separate source code and I'm merging their report into one. And the merged report shows really low coverage number.


I just created a ticket here: https://support.atlassian.com/servicedesk/customer/portal/13/CLV-6373

Please note that my company already made the purchase, I'm waiting for the license. The SEN number on the ticket for now is my trail license.


Thanks,

Will

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Published Thursday in Confluence

Three common content challenges + how to manage them

An efficient enterprise content management system, or ECM, is a must-have for companies that create work online (cough   cough, all companies). If content calendars, marketing plans, and bu...

89 views 0 6
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you