Hi I am trying to integrate gpg keys into my bitbucket pipelines to securely decrypt the files during pipelines runtime
Steps followed:
step1: Created a new gpg key pair on my local
step2: exported both private and public keys
step3: encrypted a sample file using the above key as recipient
step4: Imported the keys in to bitbucket pipelines as below
# Command used to import private key
gpg --batch --import private.key
# Command used to import public key
gpg --batch --import public.key
step5: I am able to successfully add the keys which i am able to confirm using the below command in the BB pipelines
gpg --list-secret-keys
step6: Now using the below command in the BB pipelines I am trying to decrypt the file encrypted in step3 from my local
gpg --batch --yes --passphrase=$GPG_PRIVATE_KEY_PASSPHRASE --output sample.json --decrypt sample.json.gpg
The above step is not throwing any errors but when i try to view the file by using
cat sample.json
in the next step there it is throwing error cat: sample.json: No such file or directory
Note: All the above steps are working as expected on my local even step6 is providing the output as expected.
Please let me know is this the way to use gpg within the BB pipelines or am i missing anything. Thanks.
Hi @salluri,
The error you posted makes a reference to a file sample.txt, while the file in your cat command has a json extension.
Is the cat command in your bitbucket-pipelines.yml file cat sample.json or cat sample.txt?
Kind regards,
Theodora
Hi @Theodora Boudale ,
Apologies for the confusion i am trying to test it with both sample.txt and sample.json files in my pipelines for both i am not getting the expected output as mentioned above i pasted the other error. Now i corrected that and changed to reflect the error for json file.
Please let me know if you have any further questions regarding my approach above. Thanks.
Kind regards,
Sri Akhil Varma Alluri
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi,
Thank you for the clarification.
I would suggest running the decrypt command with the --verbose flag (documentation: https://www.gnupg.org/documentation/manuals/gnupg24/gpg.1.html). The verbose output may give an indication of what is going wrong.
I would also suggest running the decrypt command without an output file, as follows:
gpg --batch --yes --passphrase=$GPG_PRIVATE_KEY_PASSPHRASE --decrypt sample.json.gpg
Does the output of this command show the content of the decrypted file?
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Theodora Boudale ,
Thanks for the response, I tried using the --verbose flag as well and it is not displaying the content of decrypt file
The verbose flag is not giving any error the output is as below in the bitbucket pipelines
gpg: using subkey <key_id> instead of primary key <key_id>
gpg: pinentry launched (573 curses 1.1.0 - - :99)
If i use the same command on my local with verbose flag it is providing the output of the file.
gpg: enabled compatibility flags:
gpg: public key is <key_id>
gpg: using subkey <key_id> instead of primary key <key_id>
gpg: encrypted with rsa4096 key, ID <key_id>, created 2023-07-14
"<key_id><email_id>"
gpg: AES256.OCB encrypted data
gpg: original file name='sample.json'
{file output}
Regards
Sri Akhil Varma Alluri
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi,
Does this Pipelines build run in our own infrastructure or are you using one of our runners?
If the build runs in our own infrastructure, I would suggest debugging it locally with Docker as per this guide:
Builds that run in our own infrastructure use Docker containers. For every step of your build, a Docker container starts based on the image you specify in your bitbucket-pipelines.yml file. The repo gets cloned on this container (unless you have disabled cloning) and then the commands of the step's script are executed.
When you debug this locally with Docker by using the same image that you have specified in your bitbucket-pipelines.yml file, you have an environment similar to the one in Pipelines. This way we can see if the issue seems to be specific to Pipelines or not.
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Theodora Boudale ,
No i am not using our own runners i am using the build that runs in environment provided by bitbucket. I am following the link sent by you to test locally but it seems like i have no access to clone the repo I am getting the below errors.
git clone git@bitbucket.org:myBBUserName/localDebugRepo.git
Cloning into 'localDebugRepo'...
You may not have access to this repository or it no longer exists in this workspace. If you think this repository exists and you have access, make sure you are authenticated.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Regards
Sri Akhil Varma Alluri
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Theodora Boudale ,
I tried testing this approach by importing the private gpg key onto another machine and followed the same steps as i am following in the bitbucket pipelines. It seems to be working on that machine as well where i am able to decrypt the file after importing.
But it seems to be not the case with bitbucket pipelines
Please find the information below that i am trying to achieve.
GPG version on my local: gpg (GnuPG) 2.4.1
But in Bitbucket pipelines i am using the below command to install gpg
- apt-get update
- apt-get install -y gnupg
# Installing gpg in pipelines
- apt-get update
- apt-get install -y gnupg
- gpg --version
# Importing the private key
- gpg --batch --import private.key
# Check if the private key is imported or not
- gpg --list-secret-keys
# Command to decrypt the file
- gpg --no-tty --batch --passphrase "$GPG_PRIVATE_KEY_PASSPHRASE" --pinentry-mode loopback --decrypt sample.json.gpg
Bitbucket image i am using to run the above pipeline is image: atlassian/default-image:3
On my local I am using the below commands to create the gpg keys and to export the secret key. Which i am storing in the repo.
##### On My Local i used below command to create the gpg keys ######
gpg --full-generate-key
where i created keys with specification RSA and RSA and with key size 4096
Then i exported the private key using the command this i am using in the above bitbucket pipeline
gpg --export-secret-key -a key_id > private.key
Regards
Akhil
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Akhil,
Apologies for the late reply, I was out of office for a couple of days.
Thank you for the information you have provided.
I would like to ask for one more thing so I can try to reproduce this:
You mentioned that you are using in the pipeline the following command:
gpg --export-secret-key -a key_id > private.key
Is key_id the private gpg key? And if so, how did you make this file available to the pipeline? Did you commit it and push it to your repo?
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Theodora Boudale ,
No problem, Yes the key_id is of the private gpg key that i created on my local and exported that secret key using the above command you mentioned in your comment.
The below command i am using on my local as mentioned in my comment above sry for any confusion.
#### On My Local i used below command to create the gpg keys ####
# Command to create the key #
gpg --full-generate-key
## where i created keys with specification RSA and RSA and with key size 4096 ##
# Command to export the key #
Then i exported the private key using the command this i am using in the above bitbucket pipeline
gpg --export-secret-key -a key_id > private.key
Yes i have committed and pushed it to repo and storing the secret key in the repo, during run time i am using that key in the pipelines to decrypt the gpg file with the commands mentioned above.
Please let me know if you have any further questions regarding the same. Thanks.
Regards
Sri Akhil Varma Alluri
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @salluri,
Thank you for the info.
I was able to reproduce this issue, both in Pipelines and also when debugging locally with Docker and using the image atlassian/default-image:3.
I was able to make it work using the image atlassian/default-image:4 and the decrypt command mentioned in this guide:
This is the bitbucket-pipelines.yml that worked for me, where MyPassphrase is a repository variable I created for the passphrase:
image: atlassian/default-image:4
pipelines:
default:
- step:
name: Build and Test
script:
- gpg --version
- gpg --batch --import public.key
- gpg --batch --import private.key
- gpg --list-secret-keys
- ls -lah
- echo "${MyPassphrase}" | PASSPHRASE="${MyPassphrase}" gpg --batch --pinentry-mode loopback --command-fd 0 -d file1.txt.gpg > file1.txt
- ls -lah
- cat file1.txt
Gpg is already installed on this image, so there is no need to reinstall it.
I am not sure why it doesn't work with atlassian/default-image:3 and whether it's an issue with the version of gpg or the older version of Ubuntu this image is using.
Does it work for you if you use a yml file similar to my own?
Edit: I changed the name of the variable to MyPassphrase to avoid confusion.
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Theodora Boudale ,
Thanks for the clarification. Yes it worked for me with the yml file similar to yours. The above command along with the new image i was able to execute the gpg decryption on my file.
image: atlassian/default-image:4
Regards
Sri Akhil Varma Alluri
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
That's good to hear Akhil, thank you for the update!
Please feel free to reach out if you ever need anything else!
Kind regards,
Theodora
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.