Per-test coverage of integration tests with maven-failsafe-plugin Edited

Hello!

I'm struggling with setting up per-test coverage for integration tests.

I have the simple setup:

  • single clover database
  • single application
  • Maven 3 build
  • all tests and servers are run on localhost
  • integration tests are run by starting WildFly with Arquillian, clover-runtime.jar is added to the deployed artifact. Integration tests are run by maven-failsafe-plugin
  • I run
    mvn clean clover:setup install 
    to build the project and run the tests.
  • The i run
    mvn clover:clover
    to generate the html report based on clover.db.

For unit-tests, per-test coverage works superbly: in the html-report on covered lines I see a green line, and by clicking on it it displays the tests that cover this line.

For integration tests, the green/red coverage is shown, but there is no green line and I can't see, which lines are covered by tests.

I tried to do instrumentation-level distributed coverage by adding

<distributedCoverage/>

to clover-maven-plugin configuration
and

<configuration>
<forkMode>once</forkMode>
<systemProperties>
<property>
<name>clover.server</name>
<value>true</value>
</property>
</systemProperties>
</configuration>

to maven-surefire-plugin configuration, according to Using Distributed Per-test Coverage page.

But nothing changes: I still do not see test coverage on classes run on the server.
Logs about running distributed coverage are written to maven log.

Running TestSuite
Distributed coverage is enabled with: name=clover.tcp.server;host=localhost;port=1198;timeout=5000;numClients=0;retryPeriod=1000
Starting distributed coverage client: name=clover.tcp.server;host=localhost;port=1198;timeout=5000;numClients=0;retryPeriod=1000

The strange thing is that when I look to WildFly log, I see similar logs too (after server start):

2017-09-26 17:35:32,895 INFO  [stdout] Distributed coverage is enabled with: name=clover.tcp.server;host=localhost;port=1198;timeout=5000;numClients=0;retryPeriod=1000

2017-09-26 17:35:32,896 INFO  [stdout] Starting distributed coverage client: name=clover.tcp.server;host=localhost;port=1198;timeout=5000;numClients=0;retryPeriod=1000

which makes me think that the remote server behaves as Clover server, not as the client.


Questions:

1) Do I have to use distributed coverage at all, since the decision matrix says that for my simple case I just have to use "showUniqueCoverage=true"? I tried to add this to maven configuration, but this does not work as well.

2) Since integration tests are run via maven-failsafe-plugin, I suppose we have to add some configuration to this plugin too.
I've seen this page, but it looks extremely complex and obsolete with aggregate reports and huge report descriptors.
How to configure maven-failsafe-plugin correctly?

3) How do I understand that distributed coverage has started successfully both for the server and client sides?

4) How to make my WildFly server work as Clover-client, not as Clover-server?

 

Thanks in advance for your answers!

Regards, DP.

1 answer

1 votes

Hey,

Starting distributed coverage client

Suggests Clover started in a client mode, not server one. Therefore I think there's some problem in initialisation on a tests side.  When Clover starts in a server mode you'll see something like that:

Starting distributed coverage service.

Second thing with concerns me is numClients property I think it should be 1 since you have single application server. Without that, Clover tests won't wait for a clients to start.

If you have 2 JVM's, one where tests are executed and second for application server then I believe you need distributed coverage in order to get per-test coverage data. 

 

The page you refer is about reports. First of all, make sure you invoke instrumented integration tests (by any means, even taking a peek at tests bytecode) 

 

Cheers

Hello, Grzegorz.

Starting distributed coverage client

Suggests Clover started in a client mode, not server one.

Oh, really, I was pretty dumb. So the server-side log is correct and it is starting in client mode.

Therefore I think there's some problem in initialisation on a tests side.

Yes, thanks for noting this. So we see, that my maven-test-side is starting in client mode. This strongly matches with my concern about running integration tests via maven-failsafe-plugin while configuration is added to maven-surefire-plugin. Do I have to add the same configuration to maven-failsafe-plugin? I'll have a try.

I also have to mention that even for unit-tests it seems that i can't telnet localhost 1198 So possibly even maven-failsafe-plugin config is not working :(

Second thing with concerns me is numClients property I think it should be 1 since you have single application server. Without that, Clover tests won't wait for a clients to start.

I don't think this can happen since Arquillian-test wait for WildFly to start. So I'm pretty sure that is not the case since the tests itself pass successfully.

I'll try with numClients = 1, anyway.

Will I see any logs on client-side or server-side when Clover-Client connects to Clover-Server?

If you have 2 JVM's, one where tests are executed and second for application server then I believe you need distributed coverage in order to get per-test coverage data. 

Thank you, at least now I know that it is impossible to do per-test-coverage for different JVMs without the distributed coverage.

First of all, make sure you invoke instrumented integration tests (by any means, even taking a peek at tests bytecode) 

I'm pretty confident that my tests are instrumented.
I see the instrumented sources in target\clover\src-test-instrumented.
I see instrumented classes in \target\test-classes.
I do not use includesTestSourceRoots = false or anything similar.
And, finally, I see the test classes in clover-coverage-html-report.

 

Thanks for answering.

Regards, DP.

Suggest an answer

Log in or Join to answer
Community showcase
Teodora [Botron]
Published Thursday in Marketplace Apps

Jira Inferno: The Nine Circles of Jira Administration Hell

If you spend enough time as a Jira admin - whether you are managing a single, mid-sized instance, a large enterprise one or juggling multiple instances at once - you will eventually find yourself in ...

446 views 1 15
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot