Problem with reports from functional tests

[Leonid Iakubov]

Hi!

I'm trying to generate html report from functional tests, but have some troubles.

About my task:

There is an web-app deployed to Tomcat. I need to cover its code with TestNG/SoapUI/Silk/etc tests.

For now I have unit-tests, in-container tests (running on server with oracle and so on) and functional tests (they just send SOAP messages to my app).

So, here are my steps:

• Instrument myapp.jar with clover :

<!-- Enable Clover -->
<!-- clover-env/ -->
<taskdef resource="cloverlib.xml" classpathref="build.class.path.src1.7"/>
<target name="with-clover" depends="clover.clean, set-clover-remote-dirs">
	<clover-setup initstring="${clover.db.dir}/clover.db" relative="true" source="1.7" debug="true" flushpolicy="threaded" flushinterval="5000">
		<!-- Filter _test methods during instrumentation -->
		<methodContext name="testng_method" regexp="(.* )?public static .*(_test)[A-Z0-9].*"/>
			
		<!-- Find all _test methods -->
		<testsources dir="${server.src.build}">
			<include name="**com/myapp/**/*.java"/>
			<testclass>
				<testmethod name="^_test.*"/> 
			</testclass>
		</testsources>
			
	</clover-setup>
</target>

Let's say ${clover.db.dir}=C:\depot\clover

• Compile sources (just using classes ant task)
• Run TestNG unit-tests
• Generate clover html report from unit-tests:

<!-- Generate Clover html report for current build -->
	<target name="clover-html-report">
		<clover-report initstring="${clover.db.dir}/clover.db" debug="true">
			<current outfile="${clover.result.dir}/html" title="Clover HTML Report" summary="true">
				<format type="html"/>
				<testresults dir="${testng.result}" includes="test-results.xml"/>
				<sourcepath>
					<pathelement path="${server.src.build}/**/com/myapp/**/*.java"/>
				</sourcepath>
			</current>
		</clover-report>
	</target>

• Deploy instrumented jar to Tomcat
• Set java argument to tomcat start script:

setlocal
set JAVA_HOME=C:\NewPlatformJDK
set CATALINA_HOME=C:\apache-tomcat-6.0.33
set JAVA_OPTS=%JAVA_OPTS% -Dclover.initstring.basedir=C:\depot\clover
cd %CATALINA_HOME%
bin\startup.bat

• After I start Tomcat I can see clover.liverec file is created in clover folder
• I want to get report just from functional test. So, I delete all coverage data from unit-tests
• Run functional test - this creates a lot of coverage files in clover folder (just the same as in case of unit-tests)
• Now, I have another build.xml which I use to start functional tests and generate html report. But I receive error:

clover-html-report:

[clover-report] Clover Version 3.1.8, built on November 13 2012 (build-873)

[clover-report] Loaded from: c:\depot\3rdparty\java\clover\clover.jar

[clover-report] Clover: Commercial License registered to ....

[clover-report] Loading coverage database from: 'c:\depot\clover\clover.db'

[clover-report] No coverage recordings found. No report will be generated.

BUILD SUCCESSFUL

So, I couldn't figure out why it doesn't see the coverage data.

My clover.jar and clover.license files are located in c:\depot\3rdparty\java\clover\clover.jar (they are used by Ant). Also there is clover.jar (the same version) in %CATALINA_HOME%\webapps\myapp\WEB-INF\lib

Could you please hint me what am I doing wrong?

Thanks!

[Marek Parfianowicz · Accepted solution]

Hi Leonid,

A problem:

[clover-report] No coverage recordings found. No report will be generated

can be related with a fact that Clover was not able to match coverage recording files to the source files.

1) The most probably a reason of this error is a <sourcepath> definition in your <clover-report> task. You shall not use wildcards - just define paths separated by a standard path separator (":" or ";"). For example:

<sourcepath>
  <pathelement path="${server.src.build}/module1:${server.src.build}/module2"/>
</sourcepath>

2) Another possibility is that source file modification time stamps are newer than coverage recording files. It might occur in case when sources were updated (e.g. due to checkout from VCS) after coverage files were recorded. You've mentioned that:

• Run functional test - this creates a lot of coverage files in clover folder (just the same as in case of unit-tests)
• Now, I have another build.xml which I use to start functional tests and generate html report.

Does it mean that you executed a build twice?

You can investigate these two cases by using Clover version 3.1.11 and running report generation with 'ant -d' option.

ad 1) The [clover-report] task will print information source files found. For example:

[clover-report] looking for c:\Work\clover-ant-3.1.11\tutorial\src\main\java\com\cenqua\samples\money\MoneyBag.java: FOUND
[clover-report] looking for c:\Work\clover-ant-3.1.11\tutorial\src\test\java_invalid\com\cenqua\samples\money\MoneyTest.java: not found
[clover-report] com/cenqua/samples/money/MoneyTest.java not found on path

ad 2) It will also print information about file time stamps (minVersion / maxVersion) and database versions (related with recording files). For example:

[clover-report] Source files timestamps:
[clover-report]   minVersion=1368009257501 (Wed May 08 12:34:17 CEST 2013)
[clover-report]   maxVersion=1368009755815 (Wed May 08 12:42:35 CEST 2013)
[clover-report] Instrumentation sessions:
[clover-report]   0: version 1368009755815 (Wed May 08 12:42:35 CEST 2013)
[clover-report]   1: version 1368009270803 (Wed May 08 12:34:30 CEST 2013)

Please check if there is no warning like:

Ignoring coverage recording .... because no FileInfo supports its coverage range

I hope this helps :-) Feel free to ask!

Cheers
Marek

[Marek Parfianowicz]

You shall see a grey check mark in the answer's top right corner. Like this green mark.

[Porsche Team]

Hi Marek,

Could you please close this question? :)

Because I don't see any option to do that (don't see any "grey check mark").

The problem was solved a long time ago, thanks to you, but Atlassian still ask me to close this question.

Thanks!

[Porsche Team]

Strange. The green mark I can see, but the grey one on current page - can't.

Checked in Chrome/FF/IE.

[Marek Parfianowicz]

Are you logged as 'Leonid Iakubov' or 'Porsche Team'?