Get unit tests coverage along with integration tests coverage

Hi,

I have created a clover-report from unit tests using coverageA.db. Can I merge this db along with my Integration tests coverageB.db? Means can I just ftp coverageA.db to the test machine and merge this with coverageB.db and create a new report or I need to ftp all the temp files (coverage.db2nvijc4wn7af_fi_6q0wyl_i0zgawrd.s, coverage.db2nviiselinwp_af_6q0wyl_i0zg9baa.s, etc) too??

My objective is to generate a clover-report which has coverage information for both unit and integration tests.

My unit tests is different than integration tests server.

Thanks,

Abhinav

1 answer

0 vote

Hi Abhinav,

Can I merge this db along with my Integration tests coverageB.db?

Yes, you can merge Clover databases. You can use the 'clover2:merge' Maven goal, <clover-merge> Ant task or the CloverMerge command line tool for this.

 

can I just ftp coverageA.db to the test machine and merge this with coverageB.db and create a new report or I need to ftp all the temp files (coverage.db2nvijc4wn7af_fi_6q0wyl_i0zgawrd.s, coverage.db2nviiselinwp_af_6q0wyl_i0zg9baa.s, etc) too??

These "temp files" are actually Clover's coverage recording files. You have to copy all of them.

Clover generates the following kinds of files:

  • Clover database (usually named 'coverage.db' or 'clover.db')
  • Global recording files (named like 'coverage.db<some_hash>' and 'coverage.db<some_hash>.1')
  • Per-test recording files (named like 'coverage.db<some_longer_hash>.s')

 

Cheers
Marek

Hi Marek, I have generated a clover-report by merging both coverageA.db and coverageB.db and generated a new coverageC.db. But the report only shows coverage info for coverageA.db (unit-tests) it doesn't take coverage info of integration tests. Why is it so? Not sure what I missed. Thanks for your reply.. Regards.

1. Could you generate a report from coverageB.db (integration tests) and check whether it shows any code coverage? If yes then: 2. When merging databases and generating report from coverageC.db (the merged one) - did you copy all three kinds of files for merging? If yes then: 3. Could you run reporting with debug logging (ant -d / mvn -X / HtmlReporter -d) and check if you see any warnings?

ad 1. In case when a report generated from coverageB.db (integration tests) shows no code coverage, then you may have some problems during coverage recording - for instance, an application may not have found the coverageB.db file. ad 2. In case you have copied per-test coverage file (*.s) but not global coverage files (coverage.db<hash_code> and coverage.db<hash_code>.1) then a report generated will not show any coverage. This is because it is not sufficient to provide per-test coverage files only. ad 3. You may find warning messages as described here: https://confluence.atlassian.com/display/CLOVERKB/Ignoring+coverage+recording+files

Hi Marek, Yes you are right. I am getting below warnings when enabled -d: [clover-report] Ignoring coverage recording CoverageRecordingTranscript.FromStream[header=Header[dbVersion=1413179664899, writeTimeStamp=1413212475829, format=0], coverageSum=3089, hitCounts.length=95261] because no FileInfo supports its coverage range [clover-report] Read header for "/home/ta230208/coverage.dbvyk5b2_i17y1qnf": Header[dbVersion=1413179783188, writeTimeStamp=1413212191512, format=0] [clover-report] Read header for "/home/ta230208/coverage.dbvyk5b2_i17y1qnf.1": Header[dbVersion=1413179783188, writeTimeStamp=1413212192168, format=0] [clover-report] [read 95261 elements as 1552 bytes with sum 303] [clover-report] Read data for file "/home/ta230208/coverage.dbvyk5b2_i17y1qnf.1": CoverageRecordingTranscript.FromStream[header=Header[dbVersion=1413179783188, writeTimeStamp=1413212192168, format=0], coverageSum=303, hitCounts.length=95261] On analyzing the same I found I am not able to generate report for Integration tests coverage db file. I have a query on this, In my process, I have enabled clover init-string then created all the clovered pkgs along with clover db (coverageB.db). After creating of pkgs I am running unit tests having different init-string and which results another clover db i.e. coverageA.db. I run integration tests on coverageB.db. Now there comes the issue, if I am generating clover report without running unit tests then the report is correct but when I create report (only for integration tests, not yet did clover-merge) after unit tests then I am getting above warnings and there is no coverage shown. Does running unit tests corrupting coverageB.db by any chance?? I think it is. Please let me know how can I resolve this conflict. Thanks again, Abhinav

> I am getting below warnings Please note that in the log excerpt above you have 'coverage.db' and not 'coverageB.db' database name. Please ensure that you name Clover databases correctly (i.e. use different names for unit tests and integration tests) > Does running unit tests corrupting coverageB.db by any chance? Yes, according to logs it seems that you have overwritten one database by another; for this reason Clover is unable to match coverage recording files to a database. Please try the following scenario first: 1. Clean workspace, build and run unit tests, generate HTML report for unit tests. Is it correct and it shows code coverage? 2. Next clean workspace, build and run integration tests, generate HTML report for integration tests. Is it correct and it shows code coverage? This will help us to confirm that code coverage recording works for both cases (i.e. that there are no other issues like deployment environment etc). As soon as you confirm that running these tests separately works fine, we can try to merge these two reports: 3. Clean workspace, build and run unit and integration tests - can you run them in a single build? are they stored in the same project? In case not, then please follow your approach with having two databases, but please ensure that you use different database names for these two test phases. After tests are finished, merge databases and create an HTML report. In case you need more support, feel free to raise a ticket on support.atlassian.com.

Hi Marek, Thanks for your valuable comments. My unit test coverage db name is: mergecoverage.db and my integration tests coverage db name is coverage.db. Yes I am able to get separate reports (above step 1 & 2) correctly but it is creating problem when unit tests and integration tests are run in single build. They are stored in same project, mergecoverage.db resides in Artifacts->clover and Artifacts->code_cov. I can not control location or init-string for mergecoverage.db as it is developed and defined by architects. Please suggest how can I use both of them without effecting each other. Regards.

> My unit test coverage db name is: mergecoverage.db How is the merged database named then? > They are stored in same project, mergecoverage.db resides in Artifacts->clover and Artifacts->code_cov Artifacts? Do you run your builds in Bamboo? If yes, then how did you define these artifacts in plan's configuration? How do you run unit and integration tests - in the same task? in the same job? in the same plan? I'm asking, because you may have a case that despite having two distinct artifact definitions they are containing the same database. > I can not control location or init-string for mergecoverage.db as it is developed and defined by architects. Could explain it a little bit? Do you have any of these permissions: - change target folder for DB file, - change name of the DB file, - provide -Dclover.initstring JVM property for tests?

Hi Marek, I using teamcity and ant build files, my build file for clover-report: <target name="clover.html" depends="cleanCloverDirs, extractCloverDirs"> <clover-merge initString="/home/ta230208/newcoverage.db"> <cloverDb initString="/home/ta230208/mergecoverage.db"/> <cloverDb initString="/home/ta230208/coverage.db" /> </clover-merge> <clover-report initString="/home/ta230208/newcoverage.db"> <current outfile="/home/ta230208/results" summary="true"> <format type="html" srcLevel="true" showEmpty="false"/> <testsources dir="test/java" excludes="**/*"/> <sourcepath> <pathelement path="/home/ta230208/source_code/Agent/src/java"/> <pathelement path="/home/ta230208/source_code/CommandLine/src/java"/> <pathelement path="/home/ta230208/source_code/Commons/src/java"/> <pathelement path="/home/ta230208/source_code/Daemon/src/java"/> <pathelement path="/home/ta230208/source_code/Install/src/java"/> <pathelement path="/home/ta230208/source_code/Failover/src/java"/> <pathelement path="/home/ta230208/source_code/TPTEx/src/java"/> </sourcepath> </current> </clover-report> </target> I cannot change anything for mergecoverage.db, I can only use it for merge or report generation. My clover setup is like: <target name="clover-setup1"> <!-- Define the clover task so we can provide code converage information for unit tests --> <copy file="${buildconfig.dir}/tools/clover/clover.license" todir="${buildconfig.dir}/lib" /> <copy file="${buildconfig.dir}/tools/clover/clover.license" todir="${base.dir}/../Open_Source/dist" /> <clover-setup flushpolicy="interval" flushinterval="1000" initString="/home/ta230208/coverage.db"> <distributedCoverage host="localhost" port="1234" numClients="0" timeout="1000"/> <fileset dir="${java.src.dir}"> <patternset refid="clover.java.src" /> </fileset> </clover-setup> </target> <target name="clover_cc" depends="clean-build,clover-setup1,compile,copy_source" description="Run test suite and populate clover code converage database." /> <target name="copy_source" description="Copy all clovered code coverage src files."> <tar destfile="Agent.tar" basedir="${src.dir}"/> <gzip destfile="Agent.tar.gz" src="Agent.tar"/> <copy file="Agent.tar.gz" todir="${teamcity.build.checkoutDir}/Artifacts/Clovered_Sources"/> </target> Regards.

<target name="clover-setup1"> ... <clover-setup flushpolicy="interval" flushinterval="1000" initString="/home/ta230208/coverage.db"> Do you use this clover-setup1 target for both unit and integration tests? If yes, then your initString is incorrect, because you're using the same path for both test phases: /home/ta230208/coverage.db.

Hi Marek, For unit tests the init-string is different location, there are some clover specific build files which does that. The above init path (/home/ta230208/coverage.db) is used while building (compiling & packaging) and running integration tests. Regards.

After the package is built and unit tests executed, I am copying mergecoverage.db along with coverage.db to /home/ta230208. Then execute integration tests. Please suggest if I am doing wrong. Regards.

This thread became quite long. May I ask you to create a ticket at support.atlassian.com? We'll continue there and later we will post an answer here as soon as the problem is resolved.

Suggest an answer

Log in or Sign up to answer
Atlassian Community Anniversary

Happy Anniversary, Atlassian Community!

This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.

Read more
Community showcase
Fadoua M. Boualem
Published Monday in Trello

Using Trello to manage events

As a Jira power user, I was at first doubtful that Trello could benefit my workflow. Jira already uses boards (ones you can customize!), so why would I even need to use Trello?! In this post you will...

492 views 8 9
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you