I'm using pipelines for testing Vue application with Cypress. It is run on self hosted Bitbucket runners. Regardless of tests result there is no report in reports tab in pull request even though it says that report is uploaded.
I'm using the following command to run tests:
npx cypress run --reporter junit --reporter-options "mochaFile=./test-reports/test-output.xml"
Relevant output follows:
Searching for test report files in directories named [test-reports, TestResults, test-results, surefire-reports, failsafe-reports] down to a depth of 4
Found matching test report file /tmp/3c6495e2-6658-52b5-95df-b5e314b63c26/build/test-reports/test-output.xml
Located test suite containing 0 tests, with 0 failures and 0 errors.
Located test suite containing 1 tests, with 1 failures and 0 errors.
Finished scanning for test reports. Found 1 test report files.
Merged test suites, total number tests is 1, with 1 failures and 0 errors.
Uploading test results
Finished uploading test results.
What can be the reason for this and is there any way to see the report in PR?
The output you included from the Build teardown of the Pipelines log is for test reporting in Pipelines. These reports mentioned in the Build teardown will not show up in pull requests, but in the Pipelines build log, in a tab named Tests, and only if you have failed tests to review.
Just in case you want to see test reports in the Pipelines log, you need to make sure that build test reports are generated in one of the supported default locations (with a directory depth of 4 levels):
The test report file scanner will begin searching from the base directory of your build: /opt/atlassian/pipelines/agent/build
This currently works with JUnit and Maven Surefire XML formats, regardless of the language they are written in.
This is documented here:
The Reports panel that you see in the details of a certain Pull request (on the right-side panel) is something completely different.
These are reports created against a commit with our API:
In order for such a report to show on the pull request page, it needs to be created for the last commit of the PR's source branch. These reports are generated automatically with some of our pipes or you could also create a pipe yourself. You can check the link below for more information:
Maybe I'm missing something. I'm using 2 steps in the pipeline to analyze code, one to execute Cypress tests and the other one to run SonarQube analysis. SonarQube creates report in the Reports tab but Cypress doesn't. Is it because after Sonar Scanner performs the analysis and uploads results to SonarQube server, SonarQube server uploads the report to the BitBucket?
Below you can see content of bitbucket-pipelines.yml.
name: Cypres E2E tests
- apt-get update
- apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
- npm ci
- npm run build
- nohup npm run start &
- npx cypress run --reporter junit --reporter-options "mochaFile=./test-reports/test-output.xml"
name: SonarQube analysis
- sonar-scanner -Dsonar.projectKey=pkey -Dsonar.sources=. -Dsonar.host.url=https://oursqserver -Dsonar.login=xyz -Dsonar.qualitygate.wait=true
Thank you for the info.
The reports I see in your screenshots concern reports created by our API, so yes, my assumption is that SonarQube uses our API to create these reports.
I am not sure how Cypress works, but if you don't see any Pull Request reports then it looks like no API calls are made to create such reports. I see in the step for Cypress the argument "mochaFile=./test-reports/test-output.xml", is that the file with the test results? If so, this report should show in the Pipelines build log in a tab named Tests, if the test results are in JUnit or Maven Surefire XML formats and if they include a fail, see an example below:
If you want to see reports in Pull Requests, you would need to look into creating them via our API on the page I listed earlier (or check if Cypress already has some sample code to do the same):