Automatic flaky test detection in Bitbucket Tests (public beta)

[#1]

We're excited to announce Flaky Test Detection in Bitbucket Pipelines! Here's what lead engineering manager Rajkumar Singh has to say:

Flaky tests erode trust in your pipeline, waste CI minutes, and slow down deployments. What if your CI could spot and silence them automatically?

In January, we launched Tests in Bitbucket Pipelines — a new way to track, organize, and understand test health right where your builds run. For the first time, teams could see test results in Bitbucket over time, drill into failures, and quarantine noisy tests — all within their repo.

This has brought great visibility and transformed how teams manage and improve test quality, but manually triaging flaky tests across thousands of cases doesn’t scale. So we took it to the next level:

Bitbucket now automatically detects flaky tests and can auto-quarantine them for you.

Read the full story on the blog: Ship Faster with Signal, Not Noise: Introducing Automatic Flaky Test Detection in Bitbucket Tests.

---

What this means for your team

No extra scripts, plugins, or dashboards. Just run your pipelines as usual, and Bitbucket does the rest:

• Flaky tests are flagged automatically: Bitbucket analyzes your test history and highlights unstable tests right in the Tests tab.

• Builds stay clean: Optionally enable auto-quarantine so the noisiest tests stop blocking your team while you fix them on your schedule.

• You stay in control: Override any test's status or quarantine setting with a click.

The result? Less time spent triaging flaky tests, so teams can focus on building instead of firefighting.

---

Getting started

New to Bitbucket Tests? Get started with the blog: [Open beta] Introducing Tests in Bitbucket Pipelines - Work Life by Atlassian.

If you're on a Standard or Premium plan and already using Pipelines:

1. Run your pipelines as usual.

2. Open the Tests tab in your repo — you'll see a Flaky score column and tests automatically flagged as flaky or quarantined.

3. Fine-tune detection and auto-quarantine in Repository settings → Tests → Detection settings.

Docs:

• Understand and manage flaky tests in Bitbucket Pipelines | Bitbucket Cloud | Atlassian Support

• Quarantining flaky tests | Bitbucket Cloud | Atlassian Support

---

Share your feedback

Automatic flaky test detection and quarantining is now live as part of our Tests feature (currently in public beta). Your feedback is key to shaping what comes next, leave a comment below or use the in-product feedback button in Bitbucket.

[Adam Sajko]

Bitbucket Pipelines still doesn't auto-cancel superseded builds when you push a new commit to the same branch.

This is a classic case of building new shiny features instead of fixing the fundamentals. Priorities.

[Dimitar Nestorov]

Who asked for this? Do you mind focusing on features we've been asking for for years in this very community?

[Rajkumar Singh]

---

Thank you @Adam Sajko  and @Dimitar Nestorov for your comment and for taking the time to share how you're using Bitbucket Pipelines — that context really helps us.

The request to automatically cancel an in‑progress pipeline when a newer one is triggered on the same branch/PR is definitely something we understand the importance of. I noticed that you've actually raised this earlier in a Community post that unfortunately hasn't gotten the attention it deserves:
Aborting/cancel previous Bitbucket pipeline execution on new Trigger.

I'll personally bring this to the attention of the team responsible and flag the existing feature request so they can look into prioritising it:
https://jira.atlassian.com/browse/BCLOUD-23526

In the meantime, if you haven't already, I'd really encourage you to try out the Tests feature in Bitbucket — it gives you test reporting, automatic flaky test detection, insights, and quarantine flows right within your repository. Seeing how it works with your real pipelines and test suites would be great, and your feedback would help us shape what we build next.

Thanks again for the thoughtful feedback and for being part of this journey with us!