Pipelines not adding bitbucket to known hosts

rob_norman-clarity

I have a repo with a pipeline that is running a sam build process to build JS lambdas. It is essentially running the following steps with the image

public.ecr.aws/sam/build-nodejs14.x:

• npm i -g npm@latest (I had to add this, or it fails with our package-lock files)
• sam build --parallel

During the build process, it is failing because it's trying to install a private repo package (I've already added SSH keys so that one can access the other) and we're seeing the error:

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://git@bitbucket.org/xxx/xxxxxxxx.git
npm ERR! command-line line 0: unsupported option "accept-new".
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.

If I attempt to run a git clone on the repo listed inside a pipeline step, I see the following:

+ git clone git@bitbucket.org:xxxx/xxxxxxxxxx.git
Cloning into 'xxxxxxxxxx'...
Warning: Permanently added the ECDSA host key for IP address '18.205.93.0' to the list of known hosts.

Which leads me to believe that the known-hosts isn't being correctly updated, but I can't add bitbucket.org as a known host as the UI tells me that it's managed internally.

Patrik S

Hello @rob_norman-clarity and thank you for reaching out to Community!

Bitbucket Pipelines indeed automatically add the bitbucket.org domain fingerprints to the known_hosts files during the build setup, and you can confirm that by including the following command in your step's script : 

cat ~/.ssh/known_hosts

You should see multiple entries for bitbucket.org domain (not individual IP addresses).

However, checking the image that you using I confirmed it comes with OpenSSH 7.4 : 

$ ssh -V

OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017

and the SSH flag CheckHostIP is enabled by default in the OpenSSH version earlier than 8.5 as indicated here and disabled by default in version 8.5 and later.

The CheckHostIP flag has the following implication in behavior of SSH :

If this flag is set to ''yes'', ssh(1) will additionally check the host IP address in the known_hosts file. This allows ssh to detect if a host key changed due to DNS spoofing. If the option is set to ''no'', the check will not be executed.

Since CheckHostIP is enabled by default in the SSH client that comes with the image you are using, when SSH client connects to Bitbucket, it also fetches the fingerprint for the Bitbucket IP that you are connecting and adds it if the IP is not added yet.

This is the reason why you are seeing the warning message when doing the clone, but this is just an informative message and it will not abort the execution of the git command nor close the ssh connection.

In that case, could you please share with us the full log output (after the Warning) of the git clone command you executed inside the pipeline step, so we can investigate further what actually caused the error?

I would also suggest using the following additional flags in the git clone command to include verbose logging, which will bring information such as what ssh key is being used :

GIT_TRACE_PACKET=1 GIT_TRACE=1 GIT_SSH_COMMAND="ssh -vvv" git clone git@bitbucket.org:<workspace>/<repository>.git

Let me know in case you have any questions.

Thank you, @rob_norman-clarity !

Patrik S