Problem getting BitBucket Pipelines to commit back version change to its own GIT repo Edited

I'm trying to enable BitBucket Pipelines for a Java-based project to auto-increment the build number following Semantic Versioning. The version incrementing is in place and working well via Maven Versions plugin, the only thing I need to get working is BitBucket Pipelines / Docker image to be able to commit back to the repository...

Here's the relevant section from my BitBucket Pipeline YAML config:

 



image: maven:3.5.2-jdk-8
pipelines:
  default:
    - step:
        caches:
          - maven
        script:
          - mvn -B -U clean install
    - step:

          #increment all code version numbers
          - snapshotversion=`grep 'build.version' build.properties | awk -F '=' '{print $2}'`
          - releaseversion=`echo "$snapshotversion" | cut -f1 -d"-"`
          - mvn versions:set -DnewVersion=$snapshotversion
          - mvn versions:commit
          - mvn -B -U clean install
                              
          # Commit any changes back to this "release/x.y.z"... definitely good to here, the rest meh :(
          - git add .
          - git config user.name "$GIT_USERNAME"
          - git config user.email "$GIT_EMAIL"
          - ssh-copy-id -i ssh-rsa $BITBUCKET_SSH_PUBLIC_KEY git@bitbucket.org:${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}.git
          - ssh -i ~/.ssh/config git@bitbucket.org:${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}.git
          - commitmsg="[skip ci] Release Candidate for merging to master branch and PROD deployment"
          - if [[ -n $(git status -s) ]] ; then filelist=`git status -s` ; git commit -a -m "$commitmsg" -m "$filelist" ; git push origin release/$releaseversion:release/$releaseversion ; else echo "No changes detected"; fi

 

 

 The error it gives is:

/usr/bin/ssh-copy-id: ERROR: failed to open ID file 'ssh-rsa.pub': No such file


So I think its fairly close, the only part I can't figure out is how to add reference to my SSH key (either by file or injecting the whole Public Key as an environment variable for instance), what I'm trying to do is clearly wrong since it errors out. Does anyone have a clear working example of committing ANYTHING back to a BitBucket GIT repo via BitBucket Pipelines?

I've followed the steps suggested here by creating an SSH Key within my BitBucket Cloud settings for the repo, but with no luck:


https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Should the SSH Key really be "automagically available" to its own repo's Pipeline, like some reported, or like others report do you need a very kludgy workaround:\

https://community.atlassian.com/t5/Bitbucket-questions/Trouble-with-SSH-and-Bitbucket-Pipelines/qaq-p/604330

https://bitbucket.org/site/master/issues/13213/push-back-to-remote-from-pipelines

Appreciate these threads as it helps to get suggestions from the community and/or Atlassian. Hoping someone can spot my stupid mistake and that last little push to CD wonderland hah... but seriously, really appreciate any support on this!

3 answers

This widget could not be displayed.

Hi Bryan,

Unfortunately a kludgy workaround is indeed currently needed.

Here's the steps to get repo push-back in Pipelines working.

  1. Set up a Pipelines SSH key as specified in Step 1 of: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html?_ga=2.166794103.859441905.1522715061-825437565.1515570924
  2. Add the public key of the SSH key you created to a Bitbucket Account's SSH keys. (Account Settings -> Security -> SSH keys). I'm unsure if there's a better way to set this up so it's not tied to an account. Otherwise you can create a dummy account if you don't want it linked to a specific team members account. Related docs here: https://confluence.atlassian.com/bitbucket/set-up-an-ssh-key-728138079.html
  3. Repo push back should now be possible. Try the following yaml.
image: ubuntu:16.04
pipelines:
default:
- step:
script:
- apt-get update -y
-
apt-get install -y git
-
echo "Hello" >> file.txt
-
git add .
-
git commit -m 'I said hello [SKIP CI]' # [SKIP CI] prevents builds being triggered off this commit.
- git push origin master

Thanks,

Phil

Thanks very much for the response @Philip Hodder I tested this out in our most recent release branch and unfortunately I'm still getting this error within BitBucket Pipelines:

Host key verification failed.fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.

Is there somewhere in specific that I need to "register" my SSH Key for once within BitBucket Pipelines for it to be aware how to commit back to its own repo?

Also, I generated the SSH Key at the repository level as the documentation seemed to show, then tried to add it to my personal account but it says:
Someone has already registered this as a deploy key.

Again I set this up under the repository itself --> Settings page is that incorrect? Do I need to use one of my own personal SSH Keys instead or anything else I might be missing?

Hi @Bryan Copeland,

I'm the new writer for Bitbucket Pipelines and I hope I can help here!

Firstly know that we are looking into this, both to improve the feature, and the documentation. We appreciate this can be a pain point. :(

So, on to an explanation:

There are 3 places you can enter an SSH key into the system, and how you decide to set that up depends on what you need to do:


1) User SSH key
( Click your face in the bottom left > Bitbucket settings > (under the Security heading) SSH keys)

A key you put in here will identify who you are, and give read and write access to all your available repositories.


2) Repository Access keys
(In your repository click Settings and under the General heading select Access keys)

Any key you put in here will have read only access to the repository (this also used to be known as a 'deploy key'). As any key entered here is only allowed to read, you can't use the same key here as you used for 1 (which has r/w). Some people never enter a key here.

 

3) Pipelines SSH key
(In your repository click Settings and under the Pipelines heading select SSH keys).

This is the key that Pipelines uses to interact with your repository and identify itself.

If you use the same key you used in 1), you'll have read and write access, BUT all the logs will say that you, Bryan, have been doing all the things that Pipelines is actually doing. Also, this key can't use a passphrase, which can be another issue if you are thinking of using your own user SSH key.

If you use the same key you've used in 2) then you'll have read only access and won't be able to push back.

Sooo, what some people do, if they want to have write access to the repo, is to make a bot Bitbucket account with access to the relevant repo. They enter in the same key (that doesn't use a passphrase) in 1) and 3) and then they have r/w access and all the logs show the bot username.

I hope that makes things a bit clearer (and if that explanation helps, I'll add it to the docs).

This widget could not be displayed.

Hi @Linette Voller thanks very much for the response, I believe I've tried all three of these approaches with no luck, but I'm going to give it another crack with the info in #2 that the "Repository Access Key", which was my latest attempt, is read-only. That was the approach I tried most recently and could explain the recent failures... I'll also try creating a new "Bot/System" user account. Although unfortunately it incurs a slightly higher monthly cost which is somewhat frustrating, the benefits in terms of man hours spent bumping the version each release will likely more than make up for the new costs.

Thanks, I'll report back here as soon as I know the outcome...

Hi @Bryan Copeland, did you get a chance to try this out in the end?

Hi @Linette Voller sorry I must be a bit daft, but I'm not understanding where to put the new Bot user's SSH key in order for Pipelines to read it, is that just within the whole project's SSH keys listing, somewhere within Pipelines, elsewhere?

Also, if I make a change as myself that creates the situation (trigger) where we want to push back to our repo (in our case just following GITFLOW so at the moment I create a "release/x.y.z" branch), how will the Bot user's account be the one attributed to bumping the version and pushing that change back to the repo?

Hi @Bryan Copeland, you're not being daft, it's a confusing area!!

Probably the most straightforward order of operations would be:

  1. Create a bot user account in BB

  2. As a repo admin, give the bot user account write permission to the repository in Repo settings > general > user and group access
  3. Go into Repo settings > Pipelines > SSH keys

  4. Generate a key pair and then copy the public key (this is the key pipelines will use to identify itself)

  5. Switch to the bot account and navigate to Bitbucket settings >Security >  SSH keys

  6. Paste public key (we are giving the bot the same ID that we are using for pipelines)


Your bot user will be credited with making the changes, as pipelines will be using the bot user's SSH key to clone your repository and perform the commands and writes. Essentially you make a trigger happen, and pipelines takes it from there.


Hope that makes things a little clearer?

@Bryan Copelanddoes the explanation above help?

@Linette Vollerthanks very much for the ongoing help with this. Apologies for the delay, but I've been waiting large 2-week gaps to retry little tweaks to the process (which is how often we cut a "release/x.y.z" release candidate branch upon which I try to bump the versions).

I thought I could just modify my existing setup and achieve what you described above, but each attempt was still falling short. For our next release, I've gone back to the drawing board and created a brand new bot account doing the exact steps you've listed.  When our next "release/x.y.z" branch gets created near the end of the week, I'll confirm whether it worked or not.

If it didn't work this time, I might be hopeless. I wonder if there's any possibility of getting a WebEx type session with yourself or someone on BitBucket support to review our setup in the worst-case?

oooh! I'll keep my fingers crossed for you!!

I'm sure we can figure out something if that doesn't work for you, but here's hoping it all works like a charm!

Thanks so much Linette, this finnnnally works!!!!

Going to be writing a blog post about Automated Versioning with Maven and Pipelines at some point soon with a link to this and your awesome advice (if you don't mind).

Thanks again!

What great news @br!!! Feel free to link and I'll also update our documentation to try and make this all a bit clearer! :)

This widget could not be displayed.

Hi all, I was able to solve this (working around my shh key using bitbucket pipelines) by creating an app_password and using that instead of my personal password.

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Aug 21, 2018 in Bitbucket

Branch Management with Bitbucket

As a project manager, I have discovered that different developers want to bring their previous branching method with them when they join the team. Some developers are used to performing individual wo...

1,347 views 8 11
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you