Help needed with test optimization in clover-ant-3.1.0

[Jacob Tredinnick]

Hi,

I can't seem to get test optimization to work with ant. In particular, I would like to run only the tests that are impacted by a change. Am I missing something?

I am using ant 1.8.2 with an eval license for the "Clover Ant - 3.1.0" product.

Using the build_optimized.xml file in the tutorial, I tried the following from a clean install:

$ ant with.clover clean test.run clover.snapshot -f build_optimized.xml

I had expected that 0 tests should run if I now do another run with no changes, but actually all (2) tests run:

$ ant with.clover clean test.run -f build_optimized.xml
Buildfile: C:\tools\clover-ant-3.1.0\tutorial\build_optimized.xml
[clover-env] Loading clover.xml: jar:file:/C:/tools/clover-ant-3.1.0/lib/clover.jar!/clover.xml

with.clover:
[clover-setup] Clover Version 3.1.0, built on May 31 2011 (build-821)
[clover-setup] Loaded from: C:\tools\clover-ant-3.1.0\lib\clover.jar
[clover-setup] Clover: Evaluation License registered to Mentor Graphics.
[clover-setup] You have 28 day(s) before your license expires.
[clover-setup] Clover is enabled with initstring 'C:\tools\clover-ant-3.1.0\tutorial\.clover\clover3_1_0.db'

clean:
[delete] Deleting directory C:\tools\clover-ant-3.1.0\tutorial\build

code.compile:
[mkdir] Created dir: C:\tools\clover-ant-3.1.0\tutorial\build\app
[javac] C:\tools\clover-ant-3.1.0\tutorial\build_optimized.xml:38: warning: 'includeantruntime' was not set, defaulting to build.sysclas
spath=last; set to false for repeatable builds
[javac] Compiling 3 source files to C:\tools\clover-ant-3.1.0\tutorial\build\app
[clover] Clover Version 3.1.0, built on May 31 2011 (build-821)
[clover] Loaded from: C:\tools\clover-ant-3.1.0\lib\clover.jar
[clover] Clover: Evaluation License registered to Mentor Graphics.
[clover] You have 28 day(s) before your license expires.
[clover] Updating existing database at 'C:\tools\clover-ant-3.1.0\tutorial\.clover\clover3_1_0.db'.
[clover] Processing files at 1.4 source level.
[clover] Clover all over. Instrumented 3 files (1 package).
[clover] Elapsed time = 0.141 secs. (21.277 files/sec, 2,028.369 srclines/sec)

test.compile:
[mkdir] Created dir: C:\tools\clover-ant-3.1.0\tutorial\build\test
[javac] C:\tools\clover-ant-3.1.0\tutorial\build_optimized.xml:46: warning: 'includeantruntime' was not set, defaulting to build.sysclas
spath=last; set to false for repeatable builds
[javac] Compiling 2 source files to C:\tools\clover-ant-3.1.0\tutorial\build\test
[clover] Clover Version 3.1.0, built on May 31 2011 (build-821)
[clover] Loaded from: C:\tools\clover-ant-3.1.0\lib\clover.jar
[clover] Clover: Evaluation License registered to Mentor Graphics.
[clover] You have 28 day(s) before your license expires.
[clover] Updating existing database at 'C:\tools\clover-ant-3.1.0\tutorial\.clover\clover3_1_0.db'.
[clover] Processing files at 1.4 source level.
[clover] Clover all over. Instrumented 2 files (1 package).
[clover] 23 test methods detected.
[clover] Elapsed time = 0.047 secs. (42.553 files/sec, 4,106.383 srclines/sec)

test.run:
[mkdir] Created dir: C:\tools\clover-ant-3.1.0\tutorial\build\testresult
[junit] Clover was unable to save any time on this optimized test run.
[junit] Clover included 2 test classes in this run (total # test classes : 2)
[junit] Running com.cenqua.samples.money.MoneyBagTest
[junit] Tests run: 22, Failures: 0, Errors: 0, Time elapsed: 0.281 sec
[junit] Running com.cenqua.samples.money.MoneyTest
[junit] Tests taking too long? Try Clover's test optimization.
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.151 sec

BUILD SUCCESSFUL
Total time: 10 seconds

[Marek Parfianowicz]

Using the build_optimized.xml file in the tutorial, I tried the following from a clean install:

ant with.clover clean test.run clover.snapshot -f build_optimized.xml

I had expected that 0 tests should run if I now do another run with no changes, but actually all (2) tests run.

I assume that you did another run using exactly the same command. This didn't work, because you have called 'ant clean', which has removed all compiled classes. As a consequence, Clover instrumented all files again, thus Clover "saw" all these files as modified. As a result, Clover chose all tests for execution.

In order to use test optimization you have to perform an incremental compilation, i.e.:

ant with.clover test.run clover.snapshot -f build_optimized.xml