Deploy repository using SSH and Pipelines

The documentation on Pipelines is shocking and nearly every post I have read, the user has had issues setting this up and ended up moving to Jenkins which gets recommended a lot. I would like to keep this in-house seeing as I'm already using Bitbucket.

I have successfully added my key to Bitbucket which allows me to post my local files to my Bitbucket repository.

 

I then enable Pipelines, create an environment variable called MY_SSH_KEY with my public ssh key, I also encoded it using the following link and pasted it into the variable.

 

I just can't get this to work and I'm on the verge of jumping ship as well as I've wasted a week on this.

 

Here's my Pipeline code

pipelines: 
    default: 
        - step: 
            script:  
                - mkdir -p ~/.ssh
                    - cat my_known_hosts >> ~/.ssh/known_hosts
                    - (umask  077 ; echo $MY_SSH_KEY | base64 --decode -i > ~/.ssh/id_rsa)
                    - ssh user@domain 'bash -s' < ./deploy.bash

The error I'm getting is:

 

+ cat my_known_hosts >> ~/.ssh/known_hosts

bash: /opt/atlassian/pipelines/agent/tmp/bashScript4585730180615321479.sh: line 16: syntax error near unexpected token `;&'

Can someone please help me to do this and not just paste a link  as it's more than likely I have already read the link.

1 answer

1 vote

Hi Daniel,

It looks like you've found some old Community questions. Sorry, we've forgotten to keep these responses up to date.

If you tell me what you're specifically trying to do I can give you a more catered answer. I've posted an updated response here: https://community.atlassian.com/t5/Bitbucket-questions/Re-How-can-I-use-SSH-in-Bitbucket-Pipelines/qaq-p/764824/comment-id/25670#M25670

Here's a copy of some of it here:

Up to date as of 04/04/2018.

For example. I want to clone repo-2 into a build from repo-1

To use SSH to clone another repository from Bitbucket you need to do the following.

  1. In repo-1 generate a Pipelines SSH key to be injected into your build container. Follow Step 1 here: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html
    • Note that if the known hosts information for bitbucket.org and github.com are injected by default. If you are using any other site you will need to configure Known Hosts, by following Step 2 in the above documentation.
  2. Add an Access Key to repo-2. Paste in the public key that was generated in the previous step.
  3. Configure your bitbucket-pipelines-yml to use git.

Can you also share some of the links you've looked at previously? I'd like to update them so other people don't run into the same issue as you. I'm in the process of updating some of the highly ranked ones that showed up on Google for me now.

Thanks,

Phil

Thank you.

 

I’m trying to deploy my bitbucket repository to my production server. I’m new to Git but I understand this is the correct way to do this.

 

as far as I understand, pipelines will deploy the updated files to your production server when a file is updated which sounds perfect. I just can’t get it to work.

Right. I think all the documentation here may be relevant for you then: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

If you're trying to deploy files to your production server, you'll probably not be using Git. You'll need to set up an SSH key and then maybe use SFTP or a CLI tool provided by your tools.

How are you currently doing deployments to your production server? What tools do you use? (Going through step by step on how you deploy at the moment would help :) )

Thank you.

I followed your link, set up a new ssh key in bitbucket, added it to my remote server, added known_hosts in bitbucket and finally edited the bitbucket-pipelines.yml with the following text:

image: ubuntu:16.04

pipelines:
default:
- step:
- apt-get update -y
- apt-get install -y git
- git clone <your_repo_here> # For example, git@bitbucket.org:username/repo-2.git

But it doesn't seem to let me commit the changes?

Your YAML is slightly incorrect. It should look like this instead:

image: ubuntu:16.04

pipelines:
default:
- step:
script:
- apt-get update -y
- apt-get install -y git
- git clone <your_repo_here> # For example, git@bitbucket.org:username/repo-2.git

You were missing the "script" section. :)

You shouldn't need to do all that git stuff, though. That was for a different specific example.

Now that you've got the ssh key and known hosts set up you should try a simpler version of your initial configuration:

pipelines: 
  default: 
    - step: 
        script:  
          - ssh user@domain 'bash -s' &lt; ./deploy.bash

Have you tried doing a deployment from your own machine. What commands are you using there? You should just be able to copy those commands into the scripts section of your pipeline.

Hi Phillip

First of all, thank you for your help.

Hopefully, this makes sense as it's 1 am here and I'm tired.

I can't seem to get my head around this....

I've used the following code:

image: ubuntu:16.04
pipelines:
default:
- step:
script:
- ssh user@domain 'bash -s' &lt; ./deploy.bash

and I get the error:

bash: ./deploy.bash: No such file or directory

I am able to ssh into my remote server / site folder from my computer using the following command on my Macbook Terminal:

ssh -t username@domain -i ~/.ssh/ssh_key "cd ~/site ; bash"

I could just use the following command but if the key is not id_rsa then it won't work and I have to point Terminal to where I've stored the key:

ssh user@domain -i ~/.ssh/ssh_key

But when I try this code in pipelines:

image: ubuntu:16.04

pipelines: 
  default: 
    - step: 
        script:
          - ssh -t user@domaink -i ~/.ssh/ssh_key "cd ~/site ; bash"

I get this error

bash: ssh: command not found

Here's what I am doing from start to finish and maybe you can spot what I'm doing wrong.

• Generate ssh key in Terminal using the following command:

ssh-keygen -t rsa -b 2048 -N '' -f ~/.ssh/macbook_key

I do this in 2048 and not 4096 just in case.

• I then copy both keys.

• I then log into my sites cPanel / SSH Access / Import keys through the browser

• I then paste the public and private keys in the boxes and authorize the public key

• I name the key 'macbook_key' to match the keys on the mac (I don't add a Passphrase)

• I then check cPanel / File Manager / .ssh : I now have my two ssh keys, I also have an authorize_key and an authorize_key2 file, 4 files in total.

• I can now ssh into my remote server by using the following command:

ssh -t username@domain -i ~/.ssh/ssh_key "cd ~/site ; bash"

What am I missing?

Thanks

For the:

bash: ssh: command not found

error. You need to install ssh before you can run it. Try adding these commands to your script.

apt-get update -y
apt-get install -y ssh

Lets see where you get after that. If things are behaving then hopefully your build will hang on the bash command. (Watch it and stop the build so you don't use too many build minutes)

If it hangs, then you'll need to add the commands you run into the SSH commands.

If it doesn't work, then it should hopefully be some authentication errors, it which case you'll need to play around and see what's missing auth.

If it's something else. We'll take it from there.

Hi Phillip

I've added the 2 lines to the script, see below:

image: ubuntu:16.04

pipelines: 
  default: 
    - step: 
        script:
- apt-get update -y
        - apt-get install -y ssh
- ssh -t user@domaink -i ~/.ssh/ssh_key "cd ~/site ; bash"

 

I'm now getting the following error:

Warning: Identity file /root/.ssh/ssh_key not accessible: No such file or directory.Pseudo-terminal will not be allocated because stdin is not a terminal.Host key verification failed.

On the bright side, it's a new error.

Still no further with this...

Looks like you have a couple errors.

The first one is that the SSH key can't be find. Can you try run SSH without "-i ~/.ssh/ssh_key", as it's not in that location. It should be located in "/root/.ssh". 

The other issue is that you won't be able to have an interactive terminal in Pipelines. (how would you interact with it after you connected?) So you will need to use protocols built on top of SSH (which still handles auth). It depends on what you're trying to do. But if you're doing file upload to another server you will probably need to use SCP or SFTP. If you're doing other stuff then it might be a bit more complicated to operate.

SCP and SFTP starting point: https://unix.stackexchange.com/questions/8707/whats-the-difference-between-sftp-scp-and-fish-protocols
SSH scripts: https://serverfault.com/questions/215756/how-do-i-run-a-local-bash-script-on-remote-machines-via-ssh

The last one is an auth failure. Most likely because the key couldn't be found earlier.

Hi Phillip

 

I have changed the code to the following:

image: ubuntu:16.04

pipelines: 
  default: 
    - step: 
        script:
- apt-get update -y
        - apt-get install -y ssh
- ssh -t user@domaink /root/.ssh "cd ~/site ; bash"

But I now get the following error:

Host key verification failed.

If I try the following code:

image: ubuntu:16.04

pipelines: 
  default: 
    - step: 
        script:
- apt-get update -y
        - apt-get install -y ssh
- ssh -t user@domaink /root/.ssh/ssh_key "cd ~/site ; bash"

I get this error:

Host key verification failed.

That means that the known_hosts contains the incorrect information for your site.

Have a look here for a more in depth explanation: https://askubuntu.com/questions/45679/ssh-connection-problem-with-host-key-verification-failed-error

You'll need to check your SSH client is accessing the correct known_hosts file on Pipelines. And that it has the correct fingerprint as your server's domain.

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Posted Jun 12, 2018 in Bitbucket

Do you use any Atlassian products for your personal projects?

After spinning my wheels trying to get organized enough to write a book for National Novel Writing Month (NaNoWriMo) I took my affinity for Atlassian products from my work life and decided to tr...

30,656 views 26 12
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you