Get unit tests coverage along with integration tests coverage

Abhinav Sahu October 8, 2014

Hi,

I have created a clover-report from unit tests using coverageA.db. Can I merge this db along with my Integration tests coverageB.db? Means can I just ftp coverageA.db to the test machine and merge this with coverageB.db and create a new report or I need to ftp all the temp files (coverage.db2nvijc4wn7af_fi_6q0wyl_i0zgawrd.s, coverage.db2nviiselinwp_af_6q0wyl_i0zg9baa.s, etc) too??

My objective is to generate a clover-report which has coverage information for both unit and integration tests.

My unit tests is different than integration tests server.

Thanks,

Abhinav

1 answer

0 votes
Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 8, 2014

Hi Abhinav,

Can I merge this db along with my Integration tests coverageB.db?

Yes, you can merge Clover databases. You can use the 'clover2:merge' Maven goal, <clover-merge> Ant task or the CloverMerge command line tool for this.

 

can I just ftp coverageA.db to the test machine and merge this with coverageB.db and create a new report or I need to ftp all the temp files (coverage.db2nvijc4wn7af_fi_6q0wyl_i0zgawrd.s, coverage.db2nviiselinwp_af_6q0wyl_i0zg9baa.s, etc) too??

These "temp files" are actually Clover's coverage recording files. You have to copy all of them.

Clover generates the following kinds of files:

  • Clover database (usually named 'coverage.db' or 'clover.db')
  • Global recording files (named like 'coverage.db<some_hash>' and 'coverage.db<some_hash>.1')
  • Per-test recording files (named like 'coverage.db<some_longer_hash>.s')

 

Cheers
Marek

Abhinav Sahu October 13, 2014

Hi Marek, I have generated a clover-report by merging both coverageA.db and coverageB.db and generated a new coverageC.db. But the report only shows coverage info for coverageA.db (unit-tests) it doesn't take coverage info of integration tests. Why is it so? Not sure what I missed. Thanks for your reply.. Regards.

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

1. Could you generate a report from coverageB.db (integration tests) and check whether it shows any code coverage? If yes then: 2. When merging databases and generating report from coverageC.db (the merged one) - did you copy all three kinds of files for merging? If yes then: 3. Could you run reporting with debug logging (ant -d / mvn -X / HtmlReporter -d) and check if you see any warnings?

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

ad 1. In case when a report generated from coverageB.db (integration tests) shows no code coverage, then you may have some problems during coverage recording - for instance, an application may not have found the coverageB.db file. ad 2. In case you have copied per-test coverage file (*.s) but not global coverage files (coverage.db<hash_code> and coverage.db<hash_code>.1) then a report generated will not show any coverage. This is because it is not sufficient to provide per-test coverage files only. ad 3. You may find warning messages as described here: https://confluence.atlassian.com/display/CLOVERKB/Ignoring+coverage+recording+files

Abhinav Sahu October 13, 2014

Hi Marek, Yes you are right. I am getting below warnings when enabled -d: [clover-report] Ignoring coverage recording CoverageRecordingTranscript.FromStream[header=Header[dbVersion=1413179664899, writeTimeStamp=1413212475829, format=0], coverageSum=3089, hitCounts.length=95261] because no FileInfo supports its coverage range [clover-report] Read header for "/home/ta230208/coverage.dbvyk5b2_i17y1qnf": Header[dbVersion=1413179783188, writeTimeStamp=1413212191512, format=0] [clover-report] Read header for "/home/ta230208/coverage.dbvyk5b2_i17y1qnf.1": Header[dbVersion=1413179783188, writeTimeStamp=1413212192168, format=0] [clover-report] [read 95261 elements as 1552 bytes with sum 303] [clover-report] Read data for file "/home/ta230208/coverage.dbvyk5b2_i17y1qnf.1": CoverageRecordingTranscript.FromStream[header=Header[dbVersion=1413179783188, writeTimeStamp=1413212192168, format=0], coverageSum=303, hitCounts.length=95261] On analyzing the same I found I am not able to generate report for Integration tests coverage db file. I have a query on this, In my process, I have enabled clover init-string then created all the clovered pkgs along with clover db (coverageB.db). After creating of pkgs I am running unit tests having different init-string and which results another clover db i.e. coverageA.db. I run integration tests on coverageB.db. Now there comes the issue, if I am generating clover report without running unit tests then the report is correct but when I create report (only for integration tests, not yet did clover-merge) after unit tests then I am getting above warnings and there is no coverage shown. Does running unit tests corrupting coverageB.db by any chance?? I think it is. Please let me know how can I resolve this conflict. Thanks again, Abhinav

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

> I am getting below warnings Please note that in the log excerpt above you have 'coverage.db' and not 'coverageB.db' database name. Please ensure that you name Clover databases correctly (i.e. use different names for unit tests and integration tests) > Does running unit tests corrupting coverageB.db by any chance? Yes, according to logs it seems that you have overwritten one database by another; for this reason Clover is unable to match coverage recording files to a database. Please try the following scenario first: 1. Clean workspace, build and run unit tests, generate HTML report for unit tests. Is it correct and it shows code coverage? 2. Next clean workspace, build and run integration tests, generate HTML report for integration tests. Is it correct and it shows code coverage? This will help us to confirm that code coverage recording works for both cases (i.e. that there are no other issues like deployment environment etc). As soon as you confirm that running these tests separately works fine, we can try to merge these two reports: 3. Clean workspace, build and run unit and integration tests - can you run them in a single build? are they stored in the same project? In case not, then please follow your approach with having two databases, but please ensure that you use different database names for these two test phases. After tests are finished, merge databases and create an HTML report. In case you need more support, feel free to raise a ticket on support.atlassian.com.

Abhinav Sahu October 13, 2014

Hi Marek, Thanks for your valuable comments. My unit test coverage db name is: mergecoverage.db and my integration tests coverage db name is coverage.db. Yes I am able to get separate reports (above step 1 & 2) correctly but it is creating problem when unit tests and integration tests are run in single build. They are stored in same project, mergecoverage.db resides in Artifacts->clover and Artifacts->code_cov. I can not control location or init-string for mergecoverage.db as it is developed and defined by architects. Please suggest how can I use both of them without effecting each other. Regards.

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

> My unit test coverage db name is: mergecoverage.db How is the merged database named then? > They are stored in same project, mergecoverage.db resides in Artifacts->clover and Artifacts->code_cov Artifacts? Do you run your builds in Bamboo? If yes, then how did you define these artifacts in plan's configuration? How do you run unit and integration tests - in the same task? in the same job? in the same plan? I'm asking, because you may have a case that despite having two distinct artifact definitions they are containing the same database. > I can not control location or init-string for mergecoverage.db as it is developed and defined by architects. Could explain it a little bit? Do you have any of these permissions: - change target folder for DB file, - change name of the DB file, - provide -Dclover.initstring JVM property for tests?

Abhinav Sahu October 13, 2014

Hi Marek, I using teamcity and ant build files, my build file for clover-report: <target name="clover.html" depends="cleanCloverDirs, extractCloverDirs"> <clover-merge initString="/home/ta230208/newcoverage.db"> <cloverDb initString="/home/ta230208/mergecoverage.db"/> <cloverDb initString="/home/ta230208/coverage.db" /> </clover-merge> <clover-report initString="/home/ta230208/newcoverage.db"> <current outfile="/home/ta230208/results" summary="true"> <format type="html" srcLevel="true" showEmpty="false"/> <testsources dir="test/java" excludes="**/*"/> <sourcepath> <pathelement path="/home/ta230208/source_code/Agent/src/java"/> <pathelement path="/home/ta230208/source_code/CommandLine/src/java"/> <pathelement path="/home/ta230208/source_code/Commons/src/java"/> <pathelement path="/home/ta230208/source_code/Daemon/src/java"/> <pathelement path="/home/ta230208/source_code/Install/src/java"/> <pathelement path="/home/ta230208/source_code/Failover/src/java"/> <pathelement path="/home/ta230208/source_code/TPTEx/src/java"/> </sourcepath> </current> </clover-report> </target> I cannot change anything for mergecoverage.db, I can only use it for merge or report generation. My clover setup is like: <target name="clover-setup1"> <!-- Define the clover task so we can provide code converage information for unit tests --> <copy file="${buildconfig.dir}/tools/clover/clover.license" todir="${buildconfig.dir}/lib" /> <copy file="${buildconfig.dir}/tools/clover/clover.license" todir="${base.dir}/../Open_Source/dist" /> <clover-setup flushpolicy="interval" flushinterval="1000" initString="/home/ta230208/coverage.db"> <distributedCoverage host="localhost" port="1234" numClients="0" timeout="1000"/> <fileset dir="${java.src.dir}"> <patternset refid="clover.java.src" /> </fileset> </clover-setup> </target> <target name="clover_cc" depends="clean-build,clover-setup1,compile,copy_source" description="Run test suite and populate clover code converage database." /> <target name="copy_source" description="Copy all clovered code coverage src files."> <tar destfile="Agent.tar" basedir="${src.dir}"/> <gzip destfile="Agent.tar.gz" src="Agent.tar"/> <copy file="Agent.tar.gz" todir="${teamcity.build.checkoutDir}/Artifacts/Clovered_Sources"/> </target> Regards.

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

<target name="clover-setup1"> ... <clover-setup flushpolicy="interval" flushinterval="1000" initString="/home/ta230208/coverage.db"> Do you use this clover-setup1 target for both unit and integration tests? If yes, then your initString is incorrect, because you're using the same path for both test phases: /home/ta230208/coverage.db.

Abhinav Sahu October 13, 2014

Hi Marek, For unit tests the init-string is different location, there are some clover specific build files which does that. The above init path (/home/ta230208/coverage.db) is used while building (compiling & packaging) and running integration tests. Regards.

Abhinav Sahu October 13, 2014

After the package is built and unit tests executed, I am copying mergecoverage.db along with coverage.db to /home/ta230208. Then execute integration tests. Please suggest if I am doing wrong. Regards.

Marek Parfianowicz
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 13, 2014

This thread became quite long. May I ask you to create a ticket at support.atlassian.com? We'll continue there and later we will post an answer here as soon as the problem is resolved.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events