No test report in pull request tab

[goran_ocokoljic]

Hi,

I'm using pipelines for testing Vue application with Cypress. It is run on self hosted Bitbucket runners. Regardless of tests result there is no report in reports tab in pull request even though it says that report is uploaded.

I'm using the following command to run tests:

npx cypress run --reporter junit --reporter-options "mochaFile=./test-reports/test-output.xml"

Relevant output follows:

Searching for test report files in directories named [test-reports, TestResults, test-results, surefire-reports, failsafe-reports] down to a depth of 4

Found matching test report file /tmp/3c6495e2-6658-52b5-95df-b5e314b63c26/build/test-reports/test-output.xml

Located test suite containing 0 tests, with 0 failures and 0 errors.

Located test suite containing 1 tests, with 1 failures and 0 errors.

Finished scanning for test reports. Found 1 test report files.

Merged test suites, total number tests is 1, with 1 failures and 0 errors.

Uploading test results

Finished uploading test results.

What can be the reason for this and is there any way to see the report in PR?

Thank you,

Goran

[Theodora Boudale · Accepted solution]

Hi Goran,

The output you included from the Build teardown of the Pipelines log is for test reporting in Pipelines. These reports mentioned in the Build teardown will not show up in pull requests, but in the Pipelines build log, in a tab named Tests, and only if you have failed tests to review.

Just in case you want to see test reports in the Pipelines log, you need to make sure that build test reports are generated in one of the supported default locations (with a directory depth of 4 levels):

./**/surefire-reports/**/*.xml
./**/failsafe-reports/**/*.xml
./**/test-results/**/*.xml
./**/test-reports/**/*.xml
./**/TestResults/**/*.xml

The test report file scanner will begin searching from the base directory of your build: /opt/atlassian/pipelines/agent/build

This currently works with JUnit and Maven Surefire XML formats, regardless of the language they are written in.

This is documented here:

• https://support.atlassian.com/bitbucket-cloud/docs/test-reporting-in-pipelines/

The Reports panel that you see in the details of a certain Pull request (on the right-side panel) is something completely different.

These are reports created against a commit with our API:

• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-reports/#api-repositories-workspace-repo-slug-commit-commit-reports-reportid-put
• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-reports/#api-repositories-workspace-repo-slug-commit-commit-reports-reportid-annotations-annotationid-put

In order for such a report to show on the pull request page, it needs to be created for the last commit of the PR's source branch. These reports are generated automatically with some of our pipes or you could also create a pipe yourself. You can check the link below for more information:

• https://support.atlassian.com/bitbucket-cloud/docs/code-insights/

Kind regards,
Theodora

[goran_ocokoljic]

Hi Theodora,

I just came back from small vacation and I haven't been able to check this out so I will do it during this week.

Again, thank you very much,

Goran

[Theodora Boudale]

Hi Goran,

Sure, take your time. Please feel free to let me know if you have any questions after you check it out.

Kind regards,
Theodora

[goran_ocokoljic]

Hi Theodora,

Maybe I'm missing something. I'm using 2 steps in the pipeline to analyze code, one to execute Cypress tests and the other one to run SonarQube analysis. SonarQube creates report in the Reports tab but Cypress doesn't. Is it because after Sonar Scanner performs the analysis and uploads results to SonarQube server, SonarQube server uploads the report to the BitBucket?

Below you can see content of bitbucket-pipelines.yml.

pipelines:
  pull-requests:
    '**':
      - step:
          name: Cypres E2E tests
          image: node:14.17.5
          runs-on:
            - self.hosted
            - linux
            - nodejs
          script:
              - apt-get update
              - apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
              - npm ci
              - npm run build
              - nohup npm run start &
              - npx cypress run --reporter junit --reporter-options "mochaFile=./test-reports/test-output.xml"
          caches:
              - cypress
              - npm
      - step:
          name: SonarQube analysis
          image: sonarsource/sonar-scanner-cli:latest
          runs-on:
              - self.hosted
              - linux
          caches:
            - sonar
          script:
            - sonar-scanner -Dsonar.projectKey=pkey -Dsonar.sources=. -Dsonar.host.url=https://oursqserver -Dsonar.login=xyz -Dsonar.qualitygate.wait=true

definitions:
  caches:
    sonar: /opt/sonar-scanner/.sonar
    npm: $HOME/.npm
    cypress: $HOME/.cache/Cypress

Thank you,

Goran

[Theodora Boudale]

Hi Goran,

SonarQube creates report in the Reports tab but Cypress doesn't.

Does SonarQube create a report in the Pipelines build log, or in the reports panel of the pull request?

Just to make sure which reports we are talking about.

Kind regards,
Theodora

[goran_ocokoljic]

Hi Theodora,

It creates report both in pipeline build log and in reports panel of the pull request. I'm sending screenshots.

Thank you very much,

Goran

[Theodora Boudale]

Hi Goran,

Thank you for the info.

The reports I see in your screenshots concern reports created by our API, so yes, my assumption is that SonarQube uses our API to create these reports.

I am not sure how Cypress works, but if you don't see any Pull Request reports then it looks like no API calls are made to create such reports. I see in the step for Cypress the argument "mochaFile=./test-reports/test-output.xml", is that the file with the test results? If so, this report should show in the Pipelines build log in a tab named Tests, if the test results are in JUnit or Maven Surefire XML formats and if they include a fail, see an example below:

If you want to see reports in Pull Requests, you would need to look into creating them via our API on the page I listed earlier (or check if Cypress already has some sample code to do the same):

https://support.atlassian.com/bitbucket-cloud/docs/code-insights/

Kind regards,
Theodora