nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Problem getting BitBucket Pipelines to commit back version change to its own GIT repo

Bryan Copeland

I'm trying to enable BitBucket Pipelines for a Java-based project to auto-increment the build number following Semantic Versioning. The version incrementing is in place and working well via Maven Versions plugin, the only thing I need to get working is BitBucket Pipelines / Docker image to be able to commit back to the repository...

Here's the relevant section from my BitBucket Pipeline YAML config:



image: maven:3.5.2-jdk-8
pipelines:
  default:
    - step: 
        caches:
          - maven
        script: 
          - mvn -B -U clean install
    - step:

          #increment all code version numbers
          - snapshotversion=`grep 'build.version' build.properties | awk -F '=' '{print $2}'`
          - releaseversion=`echo "$snapshotversion" | cut -f1 -d"-"`
          - mvn versions:set -DnewVersion=$snapshotversion
          - mvn versions:commit
          - mvn -B -U clean install
                              
          # Commit any changes back to this "release/x.y.z"... definitely good to here, the rest meh 
          - git add .
          - git config user.name "$GIT_USERNAME"
          - git config user.email "$GIT_EMAIL"
          - ssh-copy-id -i ssh-rsa $BITBUCKET_SSH_PUBLIC_KEY git@bitbucket.org:${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}.git
          - ssh -i ~/.ssh/config git@bitbucket.org:${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}.git
          - commitmsg="[skip ci] Release Candidate for merging to master branch and PROD deployment"
          - if [[ -n $(git status -s) ]] ; then filelist=`git status -s` ; git commit -a -m "$commitmsg" -m "$filelist" ; git push origin release/$releaseversion:release/$releaseversion ; else echo "No changes detected"; fi

The error it gives is:

<span>/usr/bin/ssh-copy-id: ERROR: failed to open ID file '<a href="http://ssh-rsa.pub" target="_blank" rel="nofollow noopener noreferrer">ssh-rsa.pub</a>': No such file</span>

So I think its fairly close, the only part I can't figure out is how to add reference to my SSH key (either by file or injecting the whole Public Key as an environment variable for instance), what I'm trying to do is clearly wrong since it errors out. Does anyone have a clear working example of committing ANYTHING back to a BitBucket GIT repo via BitBucket Pipelines?

I've followed the steps suggested here by creating an SSH Key within my BitBucket Cloud settings for the repo, but with no luck:

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Should the SSH Key really be "automagically available" to its own repo's Pipeline, like some reported, or like others report do you need a very kludgy workaround:\

https://community.atlassian.com/t5/Bitbucket-questions/Trouble-with-SSH-and-Bitbucket-Pipelines/qaq-p/604330

https://bitbucket.org/site/master/issues/13213/push-back-to-remote-from-pipelines

Appreciate these threads as it helps to get suggestions from the community and/or Atlassian. Hoping someone can spot my stupid mistake and that last little push to CD wonderland hah... but seriously, really appreciate any support on this!

Accepted answers

Philip Hodder

Hi Bryan,

Unfortunately a kludgy workaround is indeed currently needed.

Here's the steps to get repo push-back in Pipelines working.

Set up a Pipelines SSH key as specified in Step 1 of: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html?_ga=2.166794103.859441905.1522715061-825437565.1515570924
Add the public key of the SSH key you created to a Bitbucket Account's SSH keys. (Account Settings -> Security -> SSH keys). I'm unsure if there's a better way to set this up so it's not tied to an account. Otherwise you can create a dummy account if you don't want it linked to a specific team members account. Related docs here: https://confluence.atlassian.com/bitbucket/set-up-an-ssh-key-728138079.html
Repo push back should now be possible. Try the following yaml.

image: ubuntu:16.04
pipelines:
  default:
    - step:
        script:
          - apt-get update -y 
          - apt-get install -y git 
          - echo "Hello" >> file.txt 
          - git add . 
          - git commit -m 'I said hello [SKIP CI]' # [SKIP CI] prevents builds being triggered off this commit.
          - git push origin master

Thanks,

Phil

Bryan Copeland

Hi @Linette thanks very much for the response, I believe I've tried all three of these approaches with no luck, but I'm going to give it another crack with the info in #2 that the "Repository Access Key", which was my latest attempt, is read-only. That was the approach I tried most recently and could explain the recent failures... I'll also try creating a new "Bot/System" user account. Although unfortunately it incurs a slightly higher monthly cost which is somewhat frustrating, the benefits in terms of man hours spent bumping the version each release will likely more than make up for the new costs.

Thanks, I'll report back here as soon as I know the outcome...

All comments

Philip Hodder

Hi Bryan,

Unfortunately a kludgy workaround is indeed currently needed.

Here's the steps to get repo push-back in Pipelines working.

Set up a Pipelines SSH key as specified in Step 1 of: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html?_ga=2.166794103.859441905.1522715061-825437565.1515570924
Add the public key of the SSH key you created to a Bitbucket Account's SSH keys. (Account Settings -> Security -> SSH keys). I'm unsure if there's a better way to set this up so it's not tied to an account. Otherwise you can create a dummy account if you don't want it linked to a specific team members account. Related docs here: https://confluence.atlassian.com/bitbucket/set-up-an-ssh-key-728138079.html
Repo push back should now be possible. Try the following yaml.

image: ubuntu:16.04
pipelines:
  default:
    - step:
        script:
          - apt-get update -y 
          - apt-get install -y git 
          - echo "Hello" >> file.txt 
          - git add . 
          - git commit -m 'I said hello [SKIP CI]' # [SKIP CI] prevents builds being triggered off this commit.
          - git push origin master

Thanks,

Phil

Bryan Copeland

Thanks very much for the response @Philip Hodder I tested this out in our most recent release branch and unfortunately I'm still getting this error within BitBucket Pipelines:

<span>fatal: Could not read from remote repository.</span><span>Please make sure you have the correct access rights</span><span>and the repository exists.</span><span>Host key verification failed.</span>

Is there somewhere in specific that I need to "register" my SSH Key for once within BitBucket Pipelines for it to be aware how to commit back to its own repo?

Also, I generated the SSH Key at the repository level as the documentation seemed to show, then tried to add it to my personal account but it says:
Someone has already registered this as a deploy key.

Again I set this up under the repository itself --> Settings page is that incorrect? Do I need to use one of my own personal SSH Keys instead or anything else I might be missing?

Linette

Hi @Bryan Copeland,

I'm the new writer for Bitbucket Pipelines and I hope I can help here!

Firstly know that we are looking into this, both to improve the feature, and the documentation. We appreciate this can be a pain point.

So, on to an explanation:

There are 3 places you can enter an SSH key into the system, and how you decide to set that up depends on what you need to do:

1) User SSH key
( Click your face in the bottom left > Bitbucket settings > (under the Security heading) SSH keys)

A key you put in here will identify who you are, and give read and write access to all your available repositories.

2) Repository Access keys
(In your repository click Settings and under the General heading select Access keys)

Any key you put in here will have read only access to the repository (this also used to be known as a 'deploy key'). As any key entered here is only allowed to read, you can't use the same key here as you used for 1 (which has r/w). Some people never enter a key here.

3) Pipelines SSH key
(In your repository click Settings and under the Pipelines heading select SSH keys).

This is the key that Pipelines uses to interact with your repository and identify itself.

If you use the same key you used in 1), you'll have read and write access, BUT all the logs will say that you, Bryan, have been doing all the things that Pipelines is actually doing. Also, this key can't use a passphrase, which can be another issue if you are thinking of using your own user SSH key.

If you use the same key you've used in 2) then you'll have read only access and won't be able to push back.

Sooo, what some people do, if they want to have write access to the repo, is to make a bot Bitbucket account with access to the relevant repo. They enter in the same key (that doesn't use a passphrase) in 1) and 3) and then they have r/w access and all the logs show the bot username.

I hope that makes things a bit clearer (and if that explanation helps, I'll add it to the docs).

Bryan Copeland

Thanks, I'll report back here as soon as I know the outcome...

Linette

Hi @Bryan Copeland, did you get a chance to try this out in the end?

Bryan Copeland

Hi @Linette sorry I must be a bit daft, but I'm not understanding where to put the new Bot user's SSH key in order for Pipelines to read it, is that just within the whole project's SSH keys listing, somewhere within Pipelines, elsewhere?

Also, if I make a change as myself that creates the situation (trigger) where we want to push back to our repo (in our case just following GITFLOW so at the moment I create a "release/x.y.z" branch), how will the Bot user's account be the one attributed to bumping the version and pushing that change back to the repo?

Linette

Hi @Bryan Copeland, you're not being daft, it's a confusing area!!

Probably the most straightforward order of operations would be:

Create a bot user account in BB
As a repo admin, give the bot user account write permission to the repository in Repo settings > general > user and group access
Go into Repo settings > Pipelines > SSH keys
Generate a key pair and then copy the public key (this is the key pipelines will use to identify itself)
Switch to the bot account and navigate to Bitbucket settings >Security > SSH keys
Paste public key (we are giving the bot the same ID that we are using for pipelines)

Your bot user will be credited with making the changes, as pipelines will be using the bot user's SSH key to clone your repository and perform the commands and writes. Essentially you make a trigger happen, and pipelines takes it from there.

Hope that makes things a little clearer?

unknown

Hi all, I was able to solve this (working around my shh key using bitbucket pipelines) by creating an app_password and using that instead of my personal password.

unknown

I want to know how to set up a bot account, when I try to add a user I only get the option to do so with an email address.

Kenny Deckers

I have also looked into this and currently this seems the only way to create a bot account.

unknown

We ended up creating an AD distribution group and registering that with BitBucket; the issue for me on this is that it uses one of our licenses when there are no actual users attached.

Cory Robinson

https://support.atlassian.com/bitbucket-cloud/docs/push-back-to-your-repository/

Kerry Johnson_5ws5qo

I use a CLI tool called Lerna, and I then have Lerna run another CLI tool called gitpkg. When gitpkg would attempt to push tags to BitBucket, it would error out with the following:

fatal: could not read Username for 'https://bitbucket.org': No such device or address

I tried many solutions listed here and elsewhere with setting up keys, app passwords, etc. but could not work around this issue. So I looked into the build process and noticed that BitBucket Pipelines sets its own git credentials ahead of time:

git config user.name bitbucket-pipelines
git config user.email commits-noreply@bitbucket.org
git config push.default current
git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

I noticed that git commands issued directly (as in not issued through another CLI tool within) ran perfectly. So I had a suspicion that there might be a permission issue or some walled garden at play when Gitpkg tries to push its tags.

I tested my theory by re-issuing the git config lines, but expanding the config scope globally in hopes Gitpkg would see it

branches:
  master:
    - step:
        script:
          - git config --global user.name bitbucket-pipelines
- git config --global user.email commits-noreply@bitbucket.org
- git config --global push.default current
- git config --global http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/
- nvm install v12
- npm install
- lerna publish --yes

It worked! I know my issue might not be exactly the same as yours but maybe someone out here with the same as issue as me will come across this one day.

Edit: The above works so long as your branch isn't protected. In order to have this work with protected branches, you will need to look at "SSH Key pair managed by Bitbucket Pipelines": https://support.atlassian.com/bitbucket-cloud/docs/push-back-to-your-repository#SSH-Key-pair-managed-by-Bitbucket-Pipelines

Then set it like:

git config --global user.name bitbucket-pipelines
git config --global user.email commits-noreply@bitbucket.org
git remote set-url origin ${BITBUCKET_GIT_SSH_ORIGIN}

as in

branches:
  master:
    - step:
        script:
          - git config --global user.name bitbucket-pipelines
          - git config --global user.email commits-noreply@bitbucket.org
          - git remote set-url origin ${BITBUCKET_GIT_SSH_ORIGIN}
          - nvm install v12
          - npm install
          - lerna publish --yes

and it should push without getting the "Permission denied to update branch" error.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

bitbucket

bitbucket-cloud

bitbucket-pipelines