Running PHPUnit on a Laravel project and SonarCloud

[thn_op]

Hi everyone,

We have our projects in Bitbucket Repositories and we are using Bitbucket Pipelines to run our PHPUnit tests and allow SonarCloud scanning and validating our code for all Pull Requests.

Here’s my bitbucket-pipelines.yml file:

image: williamyeh/ansible:ubuntu18.04

pipelines:
  custom: ...
  pull-requests:
    '**':
      - step:
          name: Run tests and Generate coverage reports
          image: epcallan/php7-testing-phpunit:7.2-phpunit7
          caches:
            - composer
          script:
            - composer install --no-progress --no-suggest
            - vendor/bin/phpunit
      - step:
          name: Sonar check quality code
          script:
            - pipe: sonarsource/sonarcloud-scan:1.0.0
              variables:
                EXTRA_ARGS: -Dsonar.projectDescription=\"Project with sonarcloud-scan pipe\" -Dsonar.eslint.reportPaths=\"report.json\" -Dsonar.php.coverage.reportPaths=\"codeCoverage/coverage.xml\" -Dsonar.php.tests.reportPath=\"codeCoverage/test.xml\"
                SONAR_SCANNER_OPTS: -Xmx512m
                DEBUG: "false"

As you can see, I’m loading an image that has phpunit and Xdebug (because with other images was throwing an Error: No code coverage driver is available while doing the vendor/bin/phpunit ).

With this image the Bitbucket Pipeline did run vendor/bin/phpunit but with really different results as in my Localhost and did not run the last step.

We have written our tests and generating reports correctly on Localhost but no chance to make it work with Bitbucket Pipelines and Sonarcloud.

Here are the results on both places:

Localhost

Bitbucket Pipelines

In SonarCloud Dashboard we are having 0.0% Coverage.

coverage_latte.png1308×520 40.3 KB

Anyone know how to make this work?

Thanks,

Oscar

[Paweł Grądziel]

Hi,

I am wondering where you store your report files from `phpunit`.

I see that in `bitbucket-pipelines.yml` configuration you are running `phpunit` with no additional flags. In this case location of report file might be set in `phpunit.xml` file, like:

<logging>
    <log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
    <log type="coverage-clover" target="build/logs/clover.xml"/>
</logging>

Is it that in your case? 

Does these path are the same as the you pass to Sonar as EXTRA_ARGS?

EXTRA_ARGS <....> -Dsonar.php.coverage.reportPaths=\"codeCoverage/coverage.xml\" -Dsonar.php.tests.reportPath=\"codeCoverage/test.xml\"

[Paweł Grądziel]

The other thing I noticed in your config: keep in mind that each steps are independent and files created in one step are not available in any other. You can specify which files should be shared across steps with `artifacts` (see: https://confluence.atlassian.com/bitbucket/using-artifacts-in-steps-935389074.html)

[Paweł Grądziel]

Here is working example from my project

bitbucket-pipelines.yml

image: atlassian/default-image:2

definitions: 
  caches: 
    vendor: vendor
 

  steps:

    - step: &step-build 
        name: Build application 
        image: pawelgradziel/php73-cli-xdebug-composer:latest
        caches: 
          - composer
          - vendor
        script: 
          - composer install

    - step: &step-phpunit
        name: Run PHPUnit tests
        image: pawelgradziel/php73-cli-xdebug-composer:latest
        caches:
          - vendor
        script:
          - ./vendor/bin/phpunit --log-junit ./test-reports/phpunit.junit.xml --coverage-clover ./test-reports/phpunit.coverage.xml # for nice PHPUnit report see here: https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html
        artifacts: # defining the artifacts to be passed to each future step.
          - test-reports/phpunit**

    - step: &step-sonarcloud-scan
        name: Analyze code on SonarCloud
        script:
          - pipe: sonarsource/sonarcloud-scan:0.1.5
            variables:
              SONAR_TOKEN: ${SONAR_TOKEN}
              EXTRA_ARGS: '-Dproject.settings=sonar-project.properties' # point to sonar-project.properties if it's not in main folder

pipelines: 
  pull-requests: '**': 
    - step: *step-build 
    - step: *step-phpunit
    - step: *step-sonarcloud-scan

`phpunit` command is using `test-reports` as directory as it is recommended by Atlassian (see: https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html). With that set, you will see amount of tests executed/failed right in the Pipelines, like here:

please also notice that for `phpunit` step there is added `artifacts` config, so phpunit report files are then visible by next step (sonar scan):

artifacts: # defining the artifacts to be passed to each future step.
  - test-reports/phpunit**

phpunit.xml

<logging>
    <log type="junit" target="test-reports/phpunit.junit.xml"/>
    <log type="coverage-clover" target="test-reports/phpunit.coverage.xml"/>
</logging>

This sets default location of junit and clover reports, so when you run `phpunit` locally you don't have to provide these paths in command line. These paths in command line are still required when running `phpunit` in bitbucket-pipelines.yml

sonar-project.properties

sonar.php.coverage.reportPaths=test-reports/phpunit.coverage.xml
sonar.php.tests.reportPath=test-reports/phpunit.junit.xml

This tells sonar scan (including `sonarsource/sonarcloud-scan` pipe) where to look for report files. You can confirm if it's working fine, by looking at `sonarsource/sonarcloud-scan` pipe's output which should include these lines:

INFO: Analyzing PHPUnit test report: test-reports/phpunit.junit.xml
INFO: Analyzing PHPUnit coverage report: test-reports/phpunit.coverage.xml

With all that set, you should finally see code coverage in you Sonar Cloud!

[thn_op]

Hi Pawel and thanks for your detailed reply.

I've been some days away and could not come back to you before but have some doubts that will share with you cause also had some conversations with SonarCloud guys about the issue.

I was told by SonarCloud support guys that:

Autoscan is not available for Bitbucket right now.
Which means that having a sonar-project.properties file won’t have any effect.

And that statement confused me.

Anyway, will try with your changes and see if it works.

Kind regards and thanks!
Oscar

[thn_op]

Also,

<logging>
    <log type="junit" target="test-reports/phpunit.junit.xml"/>
    <log type="coverage-clover" target="test-reports/phpunit.clover.xml"/>
</logging>

In your phpunit.xml, you "coverage-clover" is called phpunit.clover.xml but anywhere else you call it like phpunit.coverage.xml.

As far as I understood, should be also phpunit.coverage.xml in the phpunit.xml file, isn't it?

[thn_op]

I'm getting a configuration error running Bitbucket Pipelines:

The 'caches' section in your bitbucket-pipelines.yml file contains a reference to a cache which does not exist in the caches definitions section.

Which is related to:

caches:
  - vendor

[thn_op]

I could run my tests with Bitbucket Pipelines but with same results as before:

The problem here is that the app is calling an external API using Guzzle, so the tests can't reach the API, as is not public.

I'm trying to mock Guzzle Client when tests are running.

Btw, my bitbucket-pipelines.yml looks like this right now:

image: williamyeh/ansible:ubuntu18.04

pipelines:
  pull-requests:
    '**':
      - step:
          name: Run PHPUnit tests
          image: pawelgradziel/php73-cli-xdebug-composer:latest
          caches:
            - composer
          script:
            - composer install --no-progress --no-suggest
            - vendor/bin/phpunit --log-junit ./test-reports/phpunit.junit.xml --coverage-clover ./test-reports/phpunit.coverage.xml
          artifacts: # defining the artifacts to be passed to each future step.
            - test-reports/phpunit**
      - step: &step-sonarcloud-scan
          name: Analyze code on SonarCloud
          script:
            - pipe: sonarsource/sonarcloud-scan:0.1.5
              variables:
                SONAR_TOKEN: ${SONAR_TOKEN}
                EXTRA_ARGS: '-Dproject.settings=sonar-project.properties' # point to sonar-project.properties if it's not in main folder

And still not sure that sonar-project.properties is needed for Bitbucket, this was written by a SonarSourcer from SonarSource Community:

Autoscan is not available for Bitbucket right now. So you can remove the .sonarcloud.properties file that won’t have any effect.

Any other comment is very appreciated.

[Paweł Grądziel]

I guess that failing PHP Unit tests is something you are working separately, right?

I also assume, that Sonar report still shows 0% coverage? Could you please show me what are the logs from the `sonarsource/sonarcloud-scan` pipe? Please change debug variable to "true" first for this pipe.

[thn_op]

Hi Pawel,

Just getting back to thank your answers, they really put me on the track.

At the end what was happening was that needed to mock the calls to the external service as the pipeline could not see it.

Now everything is working smooth and we are almost fully covered!

Thanks!

Oscar

[Bruno Jesus]

Man, I have the same problem (Coverage 0%, but I have coverage!).

What you did to resolve, please?

[Jadeja Rajpal Sinh]

https://medium.com/@rhjadeja004/generate-code-coverage-report-for-laravel-project-phpunit-sonarqube-575c77e37c19

Just go through this link , you will get to know how to generate code coverage report in sonarqube.