clover.xml does not contain code coverage for failing test case

[andrejev]

Hi all,

I have an example project, which contains 6 test cases: 5 passing and 1 failing. I does not found how to create per-test case report and because of that I have executed following command for every test case:

mvn clean clover:setup test -Dtest=%TEST_NAME%;
mvn clover:aggregate clover:clover;

It works fine for all test, but the failing test report does not contain a code coverage (count is always zero). When the failing test case is executed together with passing test case, the report is correct.

pom.xml:

<plugin>
    <groupId>org.openclover</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <version>4.2.0</version>
    <configuration>
        <generatePdf>false</generatePdf>
        <generateXml>true</generateXml>
        <generateHtml>true</generateHtml>
        <generateJson>false</generateJson>
    </configuration>
    <executions>
        <execution>
            <phase>pre-site</phase>
            <goals>
                <goal>instrument</goal>
            </goals>
        </execution>
    </executions>
</plugin>

1. Is it expected behaviour?

2. Does it exist some another solution to generate for every test case a new report file with code coverage? Maybe it is possible to access the clover.db using some standard library in python/java/whatever?

Best regards

[Grzegorz Lewandowski]

Hello,

The per-test coverage in simple builds is enabled when test sources are instrumented. This can be achieved by setting  includesTestSourceRoots=true, it's enabled by default so I suppose either you've turned it off or tests are in some different module. 

For more complex builds, when there's an application server and tests run outside this server (e.g. app server and REST tests) it's a little bit more complicated to set up the per tests coverage, you'll need to set up distributed per-test coverage then.

Regarding, your original question regarding failing tests. Yes, if test is failing Clover won't include its contribution to overall coverage stats. The reasoning behind this behavior is that since the test fails, its not trustworthy to say it provide test coverage (confidence) to your code. 

In case you still want to include coverage from failing tests, you can do it by:

* Setting up an reportDescriptor in Maven project

* Configuring reportDescriptor to includeFailedTestCoverage 

Cheers,

Grzegorz Lewandowski