Community
Products
Bitbucket
Questions
How set up bitbucket pipeline using a gcp private image via OIDC?

How set up bitbucket pipeline using a gcp private image via OIDC?

Hi, I am new into this, but I am trying to set up a pipeline using a docker image that is on google registry. I am having a some problems doing this and I can't find a good documentation. Can anyone help me? I already set a google identity federation and a pool on gcp.

pipelines:

default:

- step:

oidc: true

name: Build

image: gcr.io/myproject/it@sha2:123

- script:

.... I am stucked, I don't know if I have to use de UUID here or the audience url and how to do it.

2 answers

1 accepted

1 vote

Answer accepted

@lucas.ferreira Bitbucket Pipelines currently doesn't support OpenID Connect with Google container registry, it's only supported with AWS. Currently, you can only use OpenID Connect token to integrate with GCP from within the step.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

@Justin Thomas could You share the simplest example how to use Bitbucket OIDC with GCP?

Currently I have:

image: atlassian/default-image:3

pipelines:
  default:
  - parallel:
    - step: &docker-build-push
      name: Build and push images to GCR
      oidc: true
      image: google/cloud-sdk:alpine
      script:
      - echo "${BITBUCKET_STEP_OIDC_TOKEN}" > /tmp/credential-source-file.out
      - gcloud iam workload-identity-pools create-cred-config projects/${PROJECT_ID}/locations/global/workloadIdentityPools/bitbucket-pipelines/providers/bitbucket-pipelines --service-account="name@${PROJECT_ID}.iam.gserviceaccount.com" --output-file=/tmp/FILEPATH.json --credential-source-file=/tmp/credential-source-file.out --credential-source-type=text
      - gcloud auth login --cred-file=/tmp/FILEPATH.json
      - CLOUDSDK_CORE_DISABLE_PROMPTS=1 gcloud components install alpha
      - gcloud --project ${PROJECT_ID} alpha storage ls

But I get an error:

google.auth.exceptions.OAuthError: ('Error code invalid_target: The target service indicated by the "audience" parameters is invalid. This might either be because the pool or provider is disabled or deleted or because it doesn\'t exist.', '{"error":"invalid_target","error_description":"The target service indicated by the \\"audience\\" parameters is invalid. This might either be because the pool or provider is disabled or deleted or because it doesn\'t exist."}')

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Hi @Michal Czeraszkiewicz , Following is an example to integrate with GCP using OpenID Connect. Please let me know if you have any further questions.

Step 1: Creating a Workload Identity Pool

In this step, you are going to create a workload identity pool which is a container for a collection of external identities.

gcloud beta iam workload-identity-pools create bitbucket-pipelines-oidc-demo \
--location="global" \
--description="A workload identity pool for demoing oidc integration" \
--display-name="bitbucket-pipelines-oidc-demo"

Step 2: Add OIDC Provider to The Workload Identity Pool

In this step, you are going to configure Bitbucket Pipelines OIDC provider as an OIDC provider for the workload identity pool created in the last step.

gcloud beta iam workload-identity-pools providers update-oidc bitbucket-oidc-idp \
--workload-identity-pool="bitbucket-pipelines-oidc-demo" \
--issuer-uri="https://api.bitbucket.org/2.0/workspaces/{WORKSPACE}/pipelines-config/identity/oidc" \
--location="global" \
--attribute-mapping="google.subject=assertion.sub,attribute.workspace_uuid=assertion.workspaceUuid" \
--allowed-audiences="ari:cloud:bitbucket::workspace/{WORKSPACE_UUID}"

Note: workspace_uuid for audience is without curly brackets.

Step 3: Create a GCP Service Account

In the step, you are going to create a GCP service account, since external identities can't access most GCP resources directly.

gcloud iam service-accounts create bitbucket-pipelines-oidc-demo \
--display-name="bitbucket-pipelines-oidc-demo"

Note: This step can be done from GCP console.

Step 4: Allowing Bitbucket Pipelines Step to Impersonate Service Account

In this step, you are going to allow Bitbucket Pipelines Step (Using generated ID Token) to impersonate a service account.

gcloud iam service-accounts add-iam-policy-binding {SERVICE_ACCOUNT_EMAIL} \
--role="roles/iam.workloadIdentityUser" \
--member="principalSet://iam.googleapis.com/projects/{GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/bitbucket-pipelines-oidc-demo/attribute.workspace_uuid/{WORKSPACE_UUID}"

Step 5: Configure Build to Exchange OIDC ID Token for a Google Token

In the step, you are going to configure your build to exchange the OIDC ID token for a google token that can be used to access GCP resources.

Note: The bitbucket-pipelines.yml is using a shell script file to get the access token since this feature is a beta version and gcloud doesn’t have commands for doing the required steps.

bitbucket-pipelines.yml

image: google/cloud-sdk
pipelines:
   default:
       - step:
           oidc: true
           script:
               - ./getGoogleAccessToken.sh
               - export GCP_ACCESS_TOKEN=$(jq '.accessToken' accessToken.json)

getGoogleAccessToken.sh

#!/bin/bash

apt-get install -qq jq
sed -i "s/OIDC_TOKEN/$BITBUCKET_STEP_OIDC_TOKEN/g" getFederatedToken.json
curl -X POST \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @getFederatedToken.json \ 
     https://sts.googleapis.com/v1beta/token > federatedToken.json
FEDERATED_TOKEN=$(jq '.access_token' federatedToken.json)
curl -X POST \
     -H "Authorization: Bearer $FEDERATED_TOKEN" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d '{"scope": ["https://www.googleapis.com/auth/cloud-platform"]}' \
     https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/{SERVICE_ACCOUNT_EMAIL}:generateAccessToken > accessToken.json

getFederatedToken.json


{
"audience": "//iam.googleapis.com/projects/{GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/bitbucket-pipelines-oidc-demo/providers/bitbucket-oidc-idp",
"grantType": "urn:ietf:params:oauth:grant-type:token-exchange",
"requestedTokenType": "urn:ietf:params:oauth:token-type:access_token",
"scope": "https://www.googleapis.com/auth/cloud-platform",
"subjectTokenType": "urn:ietf:params:oauth:token-type:jwt",
"subjectToken": "OIDC_TOKEN"
}

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Like • Tracklab likes this

@Justin Thomas thanks a lot for the very helpful support!

I ended up with this setup (for testing GCP auth):

image: atlassian/default-image:3

pipelines:
  default:
  - parallel:
    - step: &docker-build-push
      name: Build and push images to GCR
      oidc: true
      image: google/cloud-sdk:alpine
      script:
      - gcloud --version
      - export GCP_PROJECT_NUMBER=12345678901
      - export PROJECT_NAME=my-project-name
      - export workload_identity_pool_name='bitbucket-pipelines'
      - export workload_identity_pool_provider_name='bitbucket-pipelines'
      - export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"
      - echo -n "${BITBUCKET_STEP_OIDC_TOKEN}" > /tmp/gcp_access_token.out
      - gcloud iam workload-identity-pools create-cred-config "projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${workload_identity_pool_name}/providers/${workload_identity_pool_provider_name}" --credential-source-file=/tmp/gcp_access_token.out --service-account="${SERVICE_ACCOUNT_EMAIL}" --output-file=sts-creds.json
      - export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
      - gcloud auth login --cred-file=`pwd`/sts-creds.json
      - gcloud info
      - CLOUDSDK_CORE_DISABLE_PROMPTS=1 gcloud components install alpha
      - gcloud --project "${PROJECT_NAME}" alpha storage ls gs://some-bucket/

I wish this (+ what You posted) would be in the official Atlassian documentation :-|. This would save me a lot of time.

Below my feedback to the commands You posted because it was NOT clear for me when to use the mustache brackets:

export WORKSPACE_NAME='my-workspace'
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'

# no mustache brackets
gcloud beta iam workload-identity-pools providers update-oidc bitbucket-oidc-idp \
  --workload-identity-pool="bitbucket-pipelines-oidc-demo" \
  --issuer-uri="https://api.bitbucket.org/2.0/workspaces/${WORKSPACE_NAME}/pipelines-config/identity/oidc" \
  --location="global" \
  --attribute-mapping="google.subject=assertion.sub,attribute.workspace_uuid=assertion.workspaceUuid" \
  --allowed-audiences="ari:cloud:bitbucket::workspace/${WORKSPACE_UUID}"

export GCP_PROJECT_NUMBER='12345678901'
export PROJECT_NAME='my-project-name'
export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'
gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT_EMAIL}" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects//locations/global/workloadIdentityPools/bitbucket-pipelines-oidc-demo/attribute.workspace_uuid/{${WORKSPACE_UUID}}"

Once again thanks for the fast and very helpful feedback!

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

@Justin Thomas thank You for posting the useful instructions!

I ended up with a little bit simpler version:

image: atlassian/default-image:3

pipelines:
  default:
  - parallel:
    - step: &docker-build-push
      name: Build and push images to GCR
      oidc: true
      image: google/cloud-sdk:alpine
      script:
      - export GCP_PROJECT_NUMBER=12345678901
      - export PROJECT_NAME=my-project-name

      - export workload_identity_pool_id='bitbucket-pipelines-oidc-demo'
      - export workload_identity_pool_provider_id='update-oidc bitbucket-oidc-idp'
      - export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"

      - echo -n "${BITBUCKET_STEP_OIDC_TOKEN}" > /tmp/gcp_access_token.out

      - gcloud iam workload-identity-pools create-cred-config "projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${workload_identity_pool_id}/providers/${workload_identity_pool_provider_id}" --credential-source-file=/tmp/gcp_access_token.out --service-account="${SERVICE_ACCOUNT_EMAIL}" --output-file=sts-creds.json

      - export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
      - gcloud auth login --cred-file=`pwd`/sts-creds.json
      - CLOUDSDK_CORE_DISABLE_PROMPTS=1 gcloud components install alpha
      - gcloud --project "${PROJECT_NAME}" alpha storage ls gs://some-bucket/

Below feedback to the commands which You posted - I found the usage of mustache brackets very confusing :-(...

export WORKSPACE_NAME='my-workspace'
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'
gcloud beta iam workload-identity-pools providers update-oidc bitbucket-oidc-idp \
  --workload-identity-pool="bitbucket-pipelines-oidc-demo" \
  --issuer-uri="https://api.bitbucket.org/2.0/workspaces/${WORKSPACE_NAME}/pipelines-config/identity/oidc" \
  --location="global" \
  --attribute-mapping="google.subject=assertion.sub,attribute.workspace_uuid=assertion.workspaceUuid" \
  --allowed-audiences="ari:cloud:bitbucket::workspace/${WORKSPACE_UUID}"

export GCP_PROJECT_NUMBER='12345678901'
export PROJECT_NAME='my-project-name'
export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'

gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT_EMAIL}" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects//locations/global/workloadIdentityPools/bitbucket-pipelines-oidc-demo/attribute.workspace_uuid/{${WORKSPACE_UUID}}"

I wish this + what You posted would be in the official Atlassian documentation. This would save me a lot of time.

Once again BIG thanks for the fast and very useful feedback

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

@Justin Thomas thank You for posting the useful instructions!

I ended up with a little bit simpler version:

image: atlassian/default-image:3

pipelines:
  default:
  - parallel:
    - step: &docker-build-push
      name: Build and push images to GCR
      oidc: true
      image: google/cloud-sdk:alpine
      script:
      - export GCP_PROJECT_NUMBER=12345678901
      - export PROJECT_NAME=my-project-name

      - export workload_identity_pool_id='bitbucket-pipelines-oidc-demo'
      - export workload_identity_pool_provider_id='update-oidc bitbucket-oidc-idp'
      - export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"

      - echo -n "${BITBUCKET_STEP_OIDC_TOKEN}" > /tmp/gcp_access_token.out

      - gcloud iam workload-identity-pools create-cred-config "projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${workload_identity_pool_id}/providers/${workload_identity_pool_provider_id}" --credential-source-file=/tmp/gcp_access_token.out --service-account="${SERVICE_ACCOUNT_EMAIL}" --output-file=sts-creds.json

      - export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
      - gcloud auth login --cred-file=`pwd`/sts-creds.json
      - CLOUDSDK_CORE_DISABLE_PROMPTS=1 gcloud components install alpha
      - gcloud --project "${PROJECT_NAME}" alpha storage ls gs://some-bucket/

Below feedback to the commands which You posted - I found the usage of mustache brackets very confusing :-(...

export WORKSPACE_NAME='my-workspace'
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'
gcloud beta iam workload-identity-pools providers update-oidc bitbucket-oidc-idp \
  --workload-identity-pool="bitbucket-pipelines-oidc-demo" \
  --issuer-uri="https://api.bitbucket.org/2.0/workspaces/${WORKSPACE_NAME}/pipelines-config/identity/oidc" \
  --location="global" \
  --attribute-mapping="google.subject=assertion.sub,attribute.workspace_uuid=assertion.workspaceUuid" \
  --allowed-audiences="ari:cloud:bitbucket::workspace/${WORKSPACE_UUID}"

export GCP_PROJECT_NUMBER='12345678901'
export PROJECT_NAME='my-project-name'
export SERVICE_ACCOUNT_EMAIL="sa-name@${PROJECT_NAME}.iam.gserviceaccount.com"
export WORKSPACE_UUID='bebebebe-bebe-bebe-bebe-bebebebebebe'

gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT_EMAIL}" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects//locations/global/workloadIdentityPools/bitbucket-pipelines-oidc-demo/attribute.workspace_uuid/{${WORKSPACE_UUID}}"

I wish this + what You posted would be in the official Atlassian documentation. This would save me a lot of time.

Once again BIG thanks for the fast and very useful feedback!

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

@Michal Czeraszkiewicz It's great to hear it worked. Yes, we will be adding these steps to the documentation.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Like • Roel Janken likes this

Happy to hear that this will land in the documentation!

Another feedback point is: for some unclear reason my reply was disappearing after I posted it.

I understand that this might be part of the moderation process... But it would be nice to get some feedback about this ;-) Otherwise I'm confused and frustrated that the reply I just wrote is gone!

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Like • Roel Janken likes this

Hi @Justin Thomas

I am trying to setup CICD pipeline to run terraform / Terragrunt code via bitbucket pipeline which should create resources on GCP.

I have tried the above steps however they don't work for me.

Could you please point me to official documentation on how to setup Bitbucket CICD pipeline for GCP using OIDC.

Here is the pipeline I have for testing and is not working.

image: atlassian/default-image:3

pipelines:

branches:

master:

- step: &docker-build-push

name: Build and push images to GCR

oidc: true

image: google/cloud-sdk:alpine

script:

- gcloud --version

- export GCP_PROJECT_NUMBER=xxxxxxxx

- export PROJECT_NAME=xxxxxxxxxxx

- export workload_identity_pool_name='bitbucket-pipelines-oidc-demo'

- export workload_identity_pool_provider_name='bitbucket-oidc-idp'

- export SERVICE_ACCOUNT_EMAIL="bitbucket-pipelines-oidc-demo@xxxxxxx-cicd.iam.gserviceaccount.com"

- echo -n "${BITBUCKET_STEP_OIDC_TOKEN}" > /tmp/gcp_access_token.out

- gcloud iam workload-identity-pools create-cred-config "projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${workload_identity_pool_name}/providers/${workload_identity_pool_provider_name}" --credential-source-file=/tmp/gcp_access_token.out --service-account="${SERVICE_ACCOUNT_EMAIL}" --output-file=sts-creds.json

- export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json

- gcloud auth login --cred-file=`pwd`/sts-creds.json

#- gcloud config set project xxxxxxxx

- gcloud info

- CLOUDSDK_CORE_DISABLE_PROMPTS=1 gcloud components install alpha

- gcloud storage buckets create gs://gcloud-terraform-image --project=xxxxxxx --default-storage-class=STANDARD --location=US --uniform-bucket-level-access

- gcloud --project "${PROJECT_NAME}" alpha storage ls gs://gcloud-terraform-image/

- terraform version

- terragrunt --version

And this is the error i receive while running the pipeline

+ gcloud storage buckets create gs://gcloud-terraform-image --project=xxxxxxx --default-storage-class=STANDARD --location=US --uniform-bucket-level-access

Creating gs://gcloud-terraform-image/...

ERROR: (gcloud.storage.buckets.create) There was a problem refreshing your current auth tokens: ('Unable to acquire impersonated credentials', '{\n "error": {\n "code": 403,\n "message": "Permission \'iam.serviceAccounts.getAccessToken\' denied on resource (or it may not exist).",\n "status": "PERMISSION_DENIED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.ErrorInfo",\n "reason": "IAM_PERMISSION_DENIED",\n "domain": "iam.googleapis.com",\n "metadata": {\n "permission": "iam.serviceAccounts.getAccessToken"\n }\n }\n ]\n }\n}\n')

Please run:

$ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account, run:

$ gcloud config set account ACCOUNT

to select an already authenticated account to use.

I have also tried - gcloud auth login as one of the step in the pipeline and it also fails with the same error.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

1 vote

@Justin Thomas Do you know (or does anyone else) if a BCLOUD ticket exists for this feature request yet (under https://jira.atlassian.com/browse/)? If so I'd love to know what it is so that I can chime in on it since this feature is sorely needed.

Specifically: The ability to authenticate to GCP using OIDC (Workload Identity Federation) at the top level of a `step` (via the `image:` parameter) without having to be forced to manually do it inside of the step.

Thanks!

Edit: I did find this issue, but I'm not sure if it relates to the top-level step. It also appears that I wrote a workaround (which works, but means you cannot use a private `image:` at the top step level): https://jira.atlassian.com/browse/BCLOUD-21138

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Forums

Product Q&A

Community resources

Support

Top groups

Community resources

Support

Learn

Community resources

Support

Events

Community resources

Support

How set up bitbucket pipeline using a gcp private image via OIDC?

2 answers

1 accepted

Step 1: Creating a Workload Identity Pool

Step 2: Add OIDC Provider to The Workload Identity Pool

Step 3: Create a GCP Service Account

Step 4: Allowing Bitbucket Pipelines Step to Impersonate Service Account

Step 5: Configure Build to Exchange OIDC ID Token for a Google Token

Suggest an answer

Was this helpful?

Thanks!

TAGS

Atlassian Community Events