nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

how to run "git clone" in an inline script

guy moore

I'm adding a task to a bamboo build plan.

I want to checkout code from other repositories using tags (not the latest) and the repository that will be checked out will be dynamic via using a build plan variable that will be overridden via running the build plan as "customized".

I do not want to use the standard "Source Code Checkout" task that is available to me as it is too restrictive on what I need to do.(see above sentence)

This is what I have in my inline script:

git clone ssh://git@stash.hq.viviport.com:7999/client/fidelity.git

I presumed this is what bamboo would do if I had chosen the standard "Source Code Checkout" task.

I presumed wrong, because this is what shows up in my failed build log:

Host key verification failed.
error fatal: Could not read from remote repository.

How can I mimic or leverage how bamboo gets source code without generating a set of ssh keys, etc, or using the standard "Source Code Checkout" task?

Accepted answers

Tim Crall1

Below is a writeup I wrote for pushing tags to Stash from Bamboo. It's probably similar to the link Bruno posted and should be easy to adapt for doing a manual git clone. You will have to generate SSH keys though.

-----

Bamboo does not have a way to automatically tag a commit in Git, but it is possible to do this with a little scripting.

Setup SSH Certificates

One problem is that the Bamboo build machines (agents) do not have the same SSH certificates set up as does the Bamboo server itself. Therefore in order to authenticate from the Bamboo agent, you will have to create a pair of SSH keys and add them to your repository.

Clone your repository onto a linux machine or a Macintosh and follow the steps shown below.

% git clone ssh://git@compamy.com/playg/helloworld.git

% cd helloworld

% mkdir ssh_keys

% cd ssh_keys

% ssh-keygen -t rsa -b 4096 -C "bamboo@cdk.com"

Enter file in which to save the key (/Users/crallt/.ssh/id_rsa): ./id_rsa (do not accept the default value or it will place your files in ~/.ssh and overwrite any keys already there!)

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in ./id_rsa.

Your public key has been saved in ./id_rsa.pub.

Now create a file named ssh_proxy.sh within your ssh_keys directory

% cd ssh_keys% vi ssh_proxy.sh (or editor of your choice)

paste in the following text and save the file.

#!/bin/bash
THISDIR="`dirname "$0"`"
set -vx
ssh -o StrictHostKeyChecking=no -i $THISDIR/id_rsa $1 $2 $3 $4

This creates a shell script that will launch the SSH command using the provided public key.

Add this new directory to your repository.

% git add .% git commit -m "Added SSH Keys"% git push

Now cat the contents of the id_rsa.pub file, select, and copy it to your clipboard with CNTRL-C or CMD-C as appropriate.

Go to the web interface for your Project in Stash. Click on Settings (note: you must have Administrator access on the repo to do this - if you do not, please consult a Project Administrator). Select Access Keys. Click "Add key".

Select Read/Write and paste in the contents of the id_rsa.pub file which you previously copied onto your clipboard. Click "Add key"

Script the Build Plan

Now that your SSH keys have been set up, it's time to add a script to your Bamboo build plan. Browse to your Build Plan within Bamboo and select Actions->Configure Plan. Select the stage where you want to add the tagging and click Add task. Scroll down and click on "Script"

Our script has to do three basic tasks. First, it has to configure itself to use our provided SSH keys. Then it has to apply the desired tag on the local clone of the git repository. Finally, it has to push the change back to Stash. This is complicated slightly by the fact that Bamboo does a weird kind of cloning that not retain the "origin" information. So we have to create a new remote and assign it the correct URL to push back to.

For a Linux Build Agent:

Click Add task and select "Script"

For the script body, enter something like this:

For a Windows Build Agent:

Click Add task and select "PowerShell Task"

For the script body, add something like this:

Conclusion

You will want to change line 12 ("git tag") in either script with something that puts the desired tag onto the commit - probably pulling it out an environmental variable that you've defined for that purpose.

The above script also assumes that it is the master branch that you are wanting to tag - modify accordingly otherwise.

Once you've done this, you should be able to run your build plan and tag the latest commit.

All comments

Bruno Rosa

Guy, you're right, this is not going to work. Due to limitations in Bitbucket Server (formerly Stash), Bamboo stores the ssh key for a Linked Repository against an individual user. When you use the Source Code Checkout task Bamboo will use that ssh key to checkout from the repository. This is one of the advantages of setting up an application link between both applications.

Now if you decide to use the script task to manually clone your repo, you won't be using the application link integration, instead you will be cloning the repository as if you were doing it through the command line. If you want to move forward with this using ssh protocol, you'll have to use ssh keys with no passphrase. There's a similar process for doing this outlined here: How to push to repository under SSH protocol using Script task. Although the documentation is not exactly for doing a git clone, the steps are pretty much the same.

Tim Crall1

-----

Bamboo does not have a way to automatically tag a commit in Git, but it is possible to do this with a little scripting.

Setup SSH Certificates

Clone your repository onto a linux machine or a Macintosh and follow the steps shown below.

% git clone ssh://git@compamy.com/playg/helloworld.git

% cd helloworld

% mkdir ssh_keys

% cd ssh_keys

% ssh-keygen -t rsa -b 4096 -C "bamboo@cdk.com"

Enter file in which to save the key (/Users/crallt/.ssh/id_rsa): ./id_rsa (do not accept the default value or it will place your files in ~/.ssh and overwrite any keys already there!)

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in ./id_rsa.

Your public key has been saved in ./id_rsa.pub.

Now create a file named ssh_proxy.sh within your ssh_keys directory

% cd ssh_keys% vi ssh_proxy.sh (or editor of your choice)

paste in the following text and save the file.

#!/bin/bash
THISDIR="`dirname "$0"`"
set -vx
ssh -o StrictHostKeyChecking=no -i $THISDIR/id_rsa $1 $2 $3 $4

This creates a shell script that will launch the SSH command using the provided public key.

Add this new directory to your repository.

% git add .% git commit -m "Added SSH Keys"% git push

Now cat the contents of the id_rsa.pub file, select, and copy it to your clipboard with CNTRL-C or CMD-C as appropriate.

Select Read/Write and paste in the contents of the id_rsa.pub file which you previously copied onto your clipboard. Click "Add key"

Script the Build Plan

For a Linux Build Agent:

Click Add task and select "Script"

For the script body, enter something like this:

For a Windows Build Agent:

Click Add task and select "PowerShell Task"

For the script body, add something like this:

Conclusion

The above script also assumes that it is the master branch that you are wanting to tag - modify accordingly otherwise.

Once you've done this, you should be able to run your build plan and tag the latest commit.

guy moore

1.)

Thanks Tim for your explicit answer and the time to read my question.

2.)

I just figured this out on Friday/Saturday with help from my friend Richard.

(will post it below)

3.)

What I discovered in this long process is:

that there are 2 levels of ssh authentication, the second that you describe with the generating keys, but the first level, which you have accounted for, "-o StrictHostKeyChecking=no" with that flag to ssh. That flag will generate the ~/.ssh/known_hosts file with the correct host key of the stash server that you are connecting to. Without that flag, and if the user who is running this inline script, "bamboo" in our case, did not already have the stash server's host key in it, one would get this message when running `git clone`:

Host key verification failed.

4.)

In that command line, where you have $1, $2, $3,... you can just put in: "$@".

5.)

This is my inline script, loaded with comments for myself.

It's purpose is to just grab 2 other repositories on top of the default one that was already gotten by the first default Source Code Checkout script.

This bamboo build plan has 2 customizable variables.

I removed some code that didn't matter for this purpose and changed other names for security purposes.

# Must run this build plan as Run-&gt;Customized and supply 2 parameters:
# parameter #1: client repository
# parameter #2: branch name or tag name


echo "# =============================== #"
echo "# git clone a client repository"
echo "# =============================== #"

# There are 2 protocols to get source code from Stash: http and ssh,
# and if using ssh, there are 2 levels of authentication.

# HTTP:
# Using HTTP, one has to also provide the active directory username and password, and
# has to have permission to read from Stash via Stash's user account permissions.

# Example #1:
# vmbuild does not have any read access on Stash, thus will get this error:
#   git clone http://vmbuild:&lt;password&gt;@stash.dog.com:7990/scm/client/repoName.git
#     vmbuild is a user on stash, and is not a member of any group
#     You do not have permission to access Stash.

# Example #2:
# When given a username that does not exist anywhere, you get this error:
#   git clone http://git@stash.dog.com:7999/client/repoName.git;
#     fatal: http://git@stash.dog.com:7999/client/repoName.git/info/refs not valid: is this a git repository?

# Example #3:
# This works as gmoore has read access to Stash.
#   git clone http://gmoore:&lt;password&gt;@stash.dog.com:7990/scm/client/repoName.git

# SSH:
# SSH Authentication level one:
# ssh access requires that the user who is running this build plan, "bamboo",
# has a ~/.ssh/known_hosts file with the host key of stash.dog.com in it
# otherwise, will get this error:
# 
# Example #1:
#   git clone ssh://gmoore@stash.dog.com:7999/client/repoName.git
#   git clone ssh://vmbuild@stash.dog.com:7999/client/repoName.git
#   git clone ssh://git@stash.dog.com:7999/client/repoName.git;
#     Host key verification failed.

# Note #1:
#   When using SSH, the username preceding the "@" is meaningless, and can be left out.

# Known_hosts creation method #1:
# One way to create and then populate the known_hosts file with the host key of the machine, stash.dog.com
# is, as the Linux user, bamboo, on the machine bamboo, is to ssh to stash.
#   `ssh bamboo@stash`

# that command will prompt user whether to accept the host key. Answering yes, will
# populate known_hosts with one (or more) lines with the host key.
# Continuing to supply password to log into stash is not necessary, as the known_hosts
# file is already populated prior to the password prompt.

# Known_hosts creation methond #2:
# Another way, is to run this command, and put the output into the known_hosts file:
#   ssh-keyscan -t rsa -H stash.dog.com &gt;&gt; /home/bamboo/.ssh/known_hosts

# Known_hosts creation methond #3:
# Another way is to pass this parameter when doing the ssh-override, (see below) when doing the `git clone`:
#     -o StrictHostKeyChecking=no

# SSH Authentication level two:
# Once one has made it past this first level of authentication, via the known_hosts file
# there is a second level that one has to get past.
# This requires a user, bamboo in this case, to generate a set of keys: a public key and a private key.
# Using Stash the application, associate the public key with either every single git repository,
# such as here for repoName:
#    http://stash.dog.com:7990/plugins/servlet/ssh/projects/CLIENT/repos/repoName/keys

# or associate the key at the project level,
#    http://stash.dog.com:7990/plugins/servlet/ssh/projects/CLIENT/keys

# and then use the private key in any git call that requires authentication, 
# such as a `git clone` or a `git push`

# Note: This link explains SSH identity files:
#   http://stackoverflow.com/questions/10054318/how-to-provide-username-and-password-when-run-git-clone-gitremote-git

# SSH Key generation:
# The ssh key pair was generated on machine bamboo as bamboo:
# $ ssh-keygen
# Generating public/private rsa key pair.
# Enter file in which to save the key (/home/bamboo/.ssh/id_rsa): stashAccess
# Enter passphrase (empty for no passphrase):
# Enter same passphrase again:
# Your identification has been saved in stashAccess.
# Your public key has been saved in stashAccess.pub.

# I took the contents of stashAccess.pub and put it here:
#   http://stash.dog.com:7990/plugins/servlet/ssh/projects/CLIENT/keys

# I renamed the private key file:
#   `mv stashPrivateKey stashAccessPrivateKey`


echo "ssh -o StrictHostKeyChecking=no  -i /home/bamboo/.ssh/stashAccessPrivateKey \$@;" &gt; ./sshOverride.bash;
chmod 700 ./sshOverride.bash;
export GIT_SSH=./sshOverride.bash;

# Notes: the $@ is this: -p 7999 fakeUserName@stash.dog.com git-upload-pack '/client/repoName.git'
#   -o UserKnownHostsFile=/home/bamboo/.ssh/known_hosts_works # to specify an alternative known_hosts file
#   -o StrictHostKeyChecking=no # this will create /home/bamboo/.ssh/known_hosts file (without prompting)

echo "Going to git clone $bamboo_clientRepositoryName";
git clone ssh://fakeUserName@stash.dog.com:7999/client/$bamboo_clientRepositoryName.git;

echo "Checkout the branch/tag of $bamboo_clientTagName";
cd $bamboo_clientRepositoryName;
git checkout $bamboo_clientTagName;

echo "Checking to see if we can cat out a file";
cat &lt;filename&gt;

Daniel Kühner

.removed.

Fricco

Do you know if there is a way to do this same thing but in Bitbucket Cloud?

chval

@Tim Crall1

Thank you for a very illustrative and detailed solution!

I would like to get access to the images after "For the script body, enter something like this:". I just get a link to a confluence page which is not accessible. I would appreciate very much if you can copy the images here.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

bamboo

bamboo-administration