How to integrate gpg keys with my Bitbucket Pipelines to decrypt the files during the pipelines run

salluri July 14, 2023

Hi I am trying to integrate gpg keys into my bitbucket pipelines to securely decrypt the files during pipelines runtime

 

Steps followed:

step1: Created a new gpg key pair on my local

step2: exported both private and public keys

step3: encrypted a sample file using the above key as recipient 

step4: Imported the keys in to bitbucket pipelines as below

# Command used to import private key
gpg --batch --import private.key

# Command used to import public key
gpg --batch --import public.key

 

step5: I am able to successfully add the keys which i am able to confirm using the below command in the BB pipelines

gpg --list-secret-keys

step6:  Now using the below command in the BB pipelines I am trying to decrypt the file encrypted in step3 from my local

gpg --batch --yes --passphrase=$GPG_PRIVATE_KEY_PASSPHRASE --output sample.json --decrypt sample.json.gpg

The above step is not throwing any errors but when i try to view the file by using

cat sample.json

in the next step there it is throwing error cat: sample.json: No such file or directory

 

Note: All the above steps are working as expected on my local even step6 is providing the output as expected.

 

Please let me know is this the way to use gpg within the BB pipelines or am i missing anything. Thanks.

1 answer

1 accepted

0 votes
Answer accepted
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

Hi @salluri,

The error you posted makes a reference to a file sample.txt, while the file in your cat command has a json extension.

Is the cat command in your bitbucket-pipelines.yml file cat sample.json or cat sample.txt?

Kind regards,
Theodora

salluri July 17, 2023

Hi @Theodora Boudale ,

 

Apologies for the confusion i am trying to test it with both sample.txt and sample.json files in my pipelines for both i am not getting the expected output as mentioned above i pasted the other error. Now i corrected that and changed to reflect the error for json file.   

Please let me know if you have any further questions regarding my approach above. Thanks.

 

Kind regards,

Sri Akhil Varma Alluri

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 17, 2023

Hi,

Thank you for the clarification.

I would suggest running the decrypt command with the --verbose flag (documentation: https://www.gnupg.org/documentation/manuals/gnupg24/gpg.1.html). The verbose output may give an indication of what is going wrong.

I would also suggest running the decrypt command without an output file, as follows:

gpg --batch --yes --passphrase=$GPG_PRIVATE_KEY_PASSPHRASE --decrypt sample.json.gpg

Does the output of this command show the content of the decrypted file?

Kind regards,
Theodora

salluri July 17, 2023

Hi @Theodora Boudale  ,

Thanks for the response, I tried using the --verbose flag as well  and it is not displaying the content of decrypt file

The verbose flag is not giving any error the output is as below in the bitbucket pipelines

gpg: using subkey <key_id> instead of primary key <key_id>

gpg: pinentry launched (573 curses 1.1.0 - - :99)

 

If i use the same command on my local with verbose flag it is providing the output of the file.

 

gpg: enabled compatibility flags:
gpg: public key is <key_id>
gpg: using subkey <key_id> instead of primary key <key_id>
gpg: encrypted with rsa4096 key, ID <key_id>, created 2023-07-14
"<key_id><email_id>"
gpg: AES256.OCB encrypted data
gpg: original file name='sample.json'

{file output}

 

Regards

Sri Akhil Varma Alluri        

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 18, 2023

Hi,

Does this Pipelines build run in our own infrastructure or are you using one of our runners?

If the build runs in our own infrastructure, I would suggest debugging it locally with Docker as per this guide:

Builds that run in our own infrastructure use Docker containers. For every step of your build, a Docker container starts based on the image you specify in your bitbucket-pipelines.yml file. The repo gets cloned on this container (unless you have disabled cloning) and then the commands of the step's script are executed.

When you debug this locally with Docker by using the same image that you have specified in your bitbucket-pipelines.yml file, you have an environment similar to the one in Pipelines. This way we can see if the issue seems to be specific to Pipelines or not.

Kind regards,
Theodora

salluri July 19, 2023

Hi @Theodora Boudale ,

 

No i am not using our own runners i am using the build that runs in environment provided by bitbucket. I am following the link sent by you to test locally but it seems like i have no access to clone the repo I am getting the below errors.

 

git clone git@bitbucket.org:myBBUserName/localDebugRepo.git
Cloning into 'localDebugRepo'...
You may not have access to this repository or it no longer exists in this workspace. If you think this repository exists and you have access, make sure you are authenticated.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

 

Regards

Sri Akhil Varma Alluri

salluri July 19, 2023

Hi @Theodora Boudale ,

I tried testing this approach by importing the private gpg key onto another machine and followed the same steps as i am following in the bitbucket pipelines. It seems to be working on that machine as well where i am able to decrypt the file after importing.

But it seems to be not the case with bitbucket pipelines

Please find the information below that i am trying to achieve.

GPG version on my local: gpg (GnuPG) 2.4.1

But in Bitbucket pipelines i am using the below command to install gpg

 

- apt-get update

- apt-get install -y gnupg

where it is installing version gpg (GnuPG) 2.2.19 on bitbucket pipelines

To reproduce this please find the below steps i am following in Bitbucket pipelines
# Installing gpg in pipelines

- apt-get update

- apt-get install -y gnupg

gpg --version
# Importing the private key

- gpg --batch --import private.key

# Check if the private key is imported or not

- gpg --list-secret-keys

# Command to decrypt the file 

- gpg --no-tty --batch --passphrase "$GPG_PRIVATE_KEY_PASSPHRASE" --pinentry-mode loopback --decrypt sample.json.gpg
 

Bitbucket image i am using to run the above pipeline is image: atlassian/default-image:3

 

On my local I am using the below commands to create the gpg keys and to export the secret key.  Which i am storing in the repo.

 

##### On My Local i used below command to create the gpg keys ######

gpg --full-generate-key

where i created keys with specification
RSA and RSA and with key size 4096


Then i exported the private key using the command this i am using in the above bitbucket pipeline

gpg --export-secret-key -a key_id > private.key

 

 Regards

Akhil

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 24, 2023

Hi Akhil,

Apologies for the late reply, I was out of office for a couple of days.

Thank you for the information you have provided.

I would like to ask for one more thing so I can try to reproduce this:

You mentioned that you are using in the pipeline the following command:

gpg --export-secret-key -a key_id > private.key

Is key_id the private gpg key? And if so, how did you make this file available to the pipeline? Did you commit it and push it to your repo?

Kind regards,
Theodora

salluri July 24, 2023

Hi @Theodora Boudale ,

 

No problem, Yes the key_id is of the private gpg key that i created on my local and exported that secret key using the above command you mentioned in your comment. 

The below command i am using on my local as mentioned in my comment above sry for any confusion. 

#### On My Local i used below command to create the gpg keys ####

# Command to create the key #

gpg --full-generate-key

## where i created keys with specification
RSA and RSA and with key size 4096 ##

# Command to export the key #

Then i exported the private key using the command this i am using in the above bitbucket pipeline

gpg --export-secret-key -a key_id > private.key

 

Yes i have committed and pushed it to repo and storing the secret key in the repo, during run time i am using that key in the pipelines to decrypt the gpg file with the commands mentioned above.

Please let me know if you have any further questions regarding the same. Thanks.

 

Regards

Sri Akhil Varma Alluri

Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 25, 2023

Hi @salluri,

Thank you for the info.

I was able to reproduce this issue, both in Pipelines and also when debugging locally with Docker and using the image atlassian/default-image:3.

I was able to make it work using the image atlassian/default-image:4 and the decrypt command mentioned in this guide:

This is the bitbucket-pipelines.yml that worked for me, where MyPassphrase is a repository variable I created for the passphrase:

image: atlassian/default-image:4

pipelines:
default:
- step:
name: Build and Test
script:
- gpg --version
- gpg --batch --import public.key
- gpg --batch --import private.key
- gpg --list-secret-keys
- ls -lah
- echo "${MyPassphrase}" | PASSPHRASE="${MyPassphrase}" gpg --batch --pinentry-mode loopback --command-fd 0 -d file1.txt.gpg > file1.txt
- ls -lah
- cat file1.txt

Gpg is already installed on this image, so there is no need to reinstall it.

I am not sure why it doesn't work with atlassian/default-image:3 and whether it's an issue with the version of gpg or the older version of Ubuntu this image is using.

Does it work for you if you use a yml file similar to my own?

Edit: I changed the name of the variable to MyPassphrase to avoid confusion.

Kind regards,
Theodora

Like salluri likes this
salluri July 26, 2023

Hi @Theodora Boudale ,

 

Thanks for the clarification. Yes it worked for me with the yml file similar to yours. The above command along with the new image i was able to execute the gpg decryption on my file.

image: atlassian/default-image:4

 

Regards

Sri Akhil Varma Alluri

Like Theodora Boudale likes this
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 28, 2023

That's good to hear Akhil, thank you for the update!

Please feel free to reach out if you ever need anything else!

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events