Integrate with 3rd party secret providers from Bitbucket Pipelines

[#1]

Hi all! We're excited to announce a new capability for customers using Bitbucket Pipelines for their CI/CD, with the release of our 3rd Party Secret Providers plugin capability.

Whilst Pipelines offers comprehensive functionality around managing Variables & Secrets in the product, many customers have asked for a way to take advantage of their existing dedicated secret management services when working with Pipelines. This capability will enable customers to integrate Pipelines to dedicated secret management solutions such as Hashicorp Vault, or any other secret provider tool customers are using. This capability is provided as a generic interface combined with a plugin system that allows plugins to be built for any 3rd party provider.

Note: Today we are releasing this functionality for builds executing on self-hosted runners - we plan to also make this functionality available for cloud in the coming 1-2 months. One important note is that for builds running via the self-hosted runner, requests to your secret provider are made directly from the runner agent, not from the Bitbucket Pipelines platform. This means your secrets never touch our servers or enter our network, with communication happening directly between the runner and your secret provider.

How do I get started?

To use a 3rd party secret provider, a URL for the provider endpoint must be configured in Bitbucket Pipelines. At runtime, Pipelines will call that URL, sending a pre-defined payload including the Pipelines OIDC token to the secret provider endpoint. The service hosted at that URL will need to validate the OIDC token contained in the request (instructions on how to do this can be found here), and then return a response to Pipelines containing the relevant secrets. Pipelines will expect to receive back a payload of secrets, matching a specific schema documented as an OpenAPI Spec (see below). This schema will also be documented in the readme of the sample repository.

Depending on the 3rd party provider being integrated with, a different plugin style will need to be implemented in order to map the secrets from the provider into a response that can be consumed by Pipelines.

A sample repository has been provided that demonstrates a simple API that receives a request from Pipelines, validates the OIDC token, and returns a basic set of secrets. This repository is designed to act as an example of the functionality that should be replicated for other solution-specific plugins.

Important note regarding support:

This functionality is designed for customers that are operating fairly sophisticated architectures and requires a non-trivial amount of customer-specific configuration and implementation.

As such, the functionality outlined here is NOT SUPPORTED by the Bitbucket Cloud support team. Any questions or concerns related to implementing or using this capability should be shared via the Pipelines community space.