Pulling a private repo from bitbucket in pipelines

[Ryan]

Hi!

I know this is a commonly asked question, but I couldnt solve it after reading

https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html 

https://community.atlassian.com/t5/Bitbucket-questions/quot-Warning-Permanently-added-the-RSA-host-key-for-IP-address/qaq-p/28906 

https://community.atlassian.com/t5/Bitbucket-questions/How-can-I-use-SSH-in-Bitbucket-Pipelines/qaq-p/347345 

So, I kind of need someone to point out to me where I went wrong.

What I need:
In Repo A, I am pulling Repo B. (Both are in the same team)

I have generated an SSH key in Repo A and copied over the public key as an Access key of Repo B.

On an pipeline run, this is the error i get.

    Host key verification failed.
    fatal: Could not read from remote repository.
    Please make sure you have the correct access rights
    and the repository exists.

I added my personal key and tried it out on my computer and I get prompted

    Warning: Permanently added 'bitbucket.org,18.205.93.0' (RSA) to the list of known hosts.

Not sure if that's the issue.

Here is the step i have in bitbucket-pipelines.yml

    git clone git@bitbucket.org:<name>/test.git .test

[Ryan]

Hey Ronald,

Thanks alot for your help. All works now.

[Ronald C_]

Hi Ryan,

Hope you're doing well. :)
The `Host key verification failed` error that you're seeing indicates that the ssh-agent in the build unable to recognize bitbucket.org fingerprint.

AFAIK, bitbucket.org's fingerprint will be added to the build by default unless there's custom configuration within the docker image that you're using.

Nonetheless, would you try the following workarounds and let us know how it goes?

• Add "bitbucket.org" fingerprint by navigating Repo Settings>SSH Keys>Known Host then try rerun your build.
• Try using the our default docker image - "atlassian/default-image:latest"

Regards,
Ronald

[Ryan]

Hey RonC,

I've added bitbucket.org to Known Hosts for both repos and it dosent seem to work.

Testing with atlassian/default-image:latest, got the repo cloned at the very least. I guess the only option I have left would be to build a docker in docker?

[Ronald C_]

Hey Ryan,

If you need that docker image, another workaround is to run the following command to add the Host key manually:

ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts

You may want to run "ssh -Tv git@bitbucket.org" to double-check the path to the known_hosts file that your image's ssh is using.

If it's not in "~/.ssh/known_hosts", you may need to change the command:

ssh-keyscan -H bitbucket.org >> /path/to/known_hosts

Hope this helps. :) 

[Ryan]

Hi Ron,

I just checked it out, in the docker, I have ssh but have not ran ssh-keygen so I have no known_hosts file anywhere. (verified with a `find / -name "known_hosts"`) or a ~/.ssh for that matter.

I am not familiar with good docker practices but it seems like a bad idea to have an ssh key in a docker image (unless someone could advice me otherwise)

What I tried was have the creation of ~/.ssh/known_hosts done in steps of the bitbucket-pipelines.yml before the clone and did as advised.

 First, the command "ssh -Tv git@bitbucket.org" always gave an error. It needed a user input confirmation and even on my computer it failed.

Second, since I know where my known_hosts file is, I just proceeded with adding the keyscan but the clone ultimately failed.

[Ryan]

Hey @Ronald C_

Would Secured Repository Variables be something that could help me out?

[Ronald C_]

Hi @Ryan

Using the Secured variables will allow you to store the SSH key at a specific location which the ssh-agent can locate.
It will depend on how you want to set up your build. :) 

Would you share us the output that you're seeing when you run "ssh -Tv git@bitbucket.org" and probably share us the docker image that you're using so that we can test out on our end as well?

Cheers,
Ronald 

[Ryan]

Hey Ronald, 

Sure! Have a peek at my bitbucket-pipelines.yml

image: movelai/robot-kernel:v0.0.0

pipelines:
default:
- step:
services:
- docker
script:
- mkdir -p ~/.ssh
- touch ~/.ssh/known_hosts
- ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts
- git clone <xxx> xxx

Just to be clear, I took the public key of repo A (which has this file) and added it as an access key of the other repo (which is private and wish to pull in as 'xxx')

[Ronald C_]

Hey Ryan,

Awesome! Thanks for the information.

While using your docker image, I ran the following command:

 script:
    - whoami
    - echo $HOME

I notice that the build is running as "root" user but the home directory is "/home/movel"

Based on my understanding of how the issue could happen:

• Pipelines store the Known_host and ssh config in the build's Home Directory which your docker image uses "/home/move1"
  You may run "echo $HOME" or "cd ~ && pwd" to check the home directory
• However, ssh-agent looked into "/root/.ssh" directory for the ssh key and known_host file

There are two options that we can proceed here:

1. Move the ssh "config" and "known_hosts" to "/root/.ssh"

   - mkdir -p /root/.ssh
   - mv ~/.ssh/* /root/.ssh/ 

2. or, Update the docker image to use "/root/" as the home directory instead of "home/movel"

This should help resolve the issue that you're seeing. :)

Regards,
Ronald

[Ronald C_]

Hey @Ryan

Another thing which crosses my mind which you can get it to work by running the build as "move1" user instead of "root".
Override the default user(Section) - Atlassian Documentation

This can be done with the following steps

• I run `cat /etc/passwd` to get the "move1" UID which you should see the following output:

root:x:0:0:root:/root:/bin/bash
.....
....
movel:x:1000:27::/home/movel:/bin/bash
....

•  Then get the UID - `1000` (for this case) and include in your bitbucket-pipelines.yml

  image: 
    name: movelai/robot-kernel:v0.0.0
    run-as-user: 1000

The above can serve as another workaround for your build :)

Cheers,
Ronald