How to get failed test count?

Question

In the build teardown step the pipeline look up for tests reports and calculate the total number of tests and the number of failures. Merged test suites, total number tests is 14, with 2 failures and 0 errors. Uploading test results Finished uploading test results. Finished scanning for test reports. Found 12 test report files. Is there any way to get or export that number in a variable? Like $BUILD_STATUS ? Thanks.

Caroline R · Answer

Hi, Fernando!

Thank you for sharing the way you solved the issue. Regarding the feature request you mentioned, I would suggest that you add your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features, and you are more than welcome to leave any feedback. You can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

In case you would like to read more about our Feature Request Policy, you can access the following link:

* Implementation of New Features Policy

Kind regards, 
Caroline

Fernando Vera_l4t4e6 · Answer

Hi @Caroline R ! Thanks for you answer.

I found another way to do it. I generate a file in my Gradle script with the tests results, then I read the file content in my pipeline script:

tasks.withType(Test) {
    testLogging {
        afterSuite { desc, result ->
if (!desc.parent) {
if (result.failedTestCount > 0) {
def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)"
                    println(output)
new File("${rootDir}/test", "result.log").text = output
                }
            }
        }
    }
}

Then the pipeline script is something like:

- step:
name: Send Slack notification
conditional:
changesets:
includePaths:
          - test/**
script:
      - export TEST_RESULTS=$(cat test/result.log)
      - export REPORT_PATH=$(cat report.log)
      - pipe: atlassian/slack-notify:2.0.0
variables:
WEBHOOK_URL: $SLACK_WEBHOOK_URL
PRETEXT: "Acceptance Tests ran for $BITBUCKET_REPO_SLUG, branch: $BITBUCKET_BRANCH"
          MESSAGE: "$TEST_RESULTS

$REPORT_PATH"

The thing is that the report.log file is always present in the project with 0 bytes, and is only written when there are failed tests. If not, the file has no changes. But the conditional execution of the pipe doesn't work, and it's always sending the slack message...  There is actually a requested featured for this but nobody is working on it.

Regards.

Caroline R · Answer

Hi, @Fernando Vera, thank you for reaching out to Atlassian Community.

Currently, it's not possible to get or export that number in a variable, however, you could parse this XML and extract this information, since the files are available inside the build. Basically, you would do what the build teardown does, but in your script.

With that said, I would also like to offer you to create a Feature Request to expose that as an environment variable, if you want.

Please feel free to share any additional questions regarding this case.

Kind regards, 
Caroline