git fetch does not work in bitbucket pipeline

Question

While configuring bitbucket pipelines to deploy repo in google instance via ssh I noticed that git fetch does take lot of time or does not even complete. Does anyone know why it happens ? Snippet of bitbucket command COMMAND: > cd ../../services; if # something ; then sudo git clone < clone link> && cd sudo git fetch -a && sudo git checkout $BITBUCKET_BRANCH && sudo docker compose up -d --build Eventually I commented out fetch command and it still worked. I figured out that during fetch it expects bitbucket password. Can someone tell me how to provide password ? I have tried URLS for fetch as well but still does not work. The only way I see is the delete the repo and recreate it each time.

Theodora Boudale · Answer

Hi @Naveen Joseph Gabriel,

Manually creating the repo in the instance shouldn't be an issue.

1. If the repo now exists in the Google instance, can you navigate to the repo directory (outside Pipelines) and run the command

git remote -v

Does the URL in the output start with git@bitbucket.org or with https:// ? This is to confirm which type of authentication is used.

2. Can you post the full output of the git fetch command here (after sanitizing the actual repo name and workspace id)? If you use an SSH key pair without a passphrase, you shouldn't get asked for a password. I would like to see if the password prompt comes for bitbucket.org or if perhaps it's a prompt to give the password of the root user, since you are using sudo.

3. Is the $SSH_USER the root user or a different one?

In case they are different, which user did you use when you created the SSH keys in the Google instance? If you created the keys for $SSH_USER and $SSH_USER is not root, I don't believe that the SSH keys will be used, as with sudo the command will try to find SSH keys for the root user.

Also could you reaffirm if deploying directly into instance using ssh is best practise ?

I'm not sure if I understand the question. Do you mean if deploying via SSH is best instead of using HTTPS? Or if cloning the repo on the instance is ok?

If you do need a Git repo on this server, then what you're doing is ok. You need to be mindful though that the clone includes the repo's commits. It is possible to also deploy with rsync, scp, ftp.  If you don't want the .git folder on the Google instance, rsync, for example, will allow you to exclude it and copy only the source code files.

if you don't want the history of the repo but only the source code files; these tools will only copy the source code and not the Git repo.

It all depends on your requirements and it is not something I can give you a definitive answer for. If you have a systems architect in your company, I believe they would be the person to ask.

Kind regards,
Theodora

Edit: The info I posted earlier about rsync, scp, and ftp was incorrect, I just edited my response and corrected it to avoid confusion.

Naveen Joseph Gabriel · Answer

Hi @Theodora Boudale

SSH keys were already set up and public key was kept in bitbucket repo.

So here is my doubt.

I had manually created the repo in the instance before I created the pipelines. So now if I I use fetch command from bitbucket pipeline it is not using ssh keys because I am assuming it ask for the password because execution of sudo git fetch -a does not gets completed.

Do you think manual creation of repo initially somehow prevents bitbucket pipelines to not use use ssh. If so, could you clarify what could be happening ?

Also, I had placed condition in command that if the directory already exist then don't run git clone command.

Also could you reaffirm if deploying directly into instance using ssh is best practise ?

Theodora Boudale · Answer

Hi @Naveen Joseph Gabriel,

Thank you for the info.

If you clone the repo every time, you don't need to do a git fetch as well, because a clone will fetch all the references.

However, if you want to clone only once and then remove the git clone command from the script and do a git fetch only:

Since you are asked for a password, I assume that you use HTTPS to clone the repo?

If so, I would recommend setting up authentication with SSH keys instead, using an SSH key pair that doesn't have a passphrase. In this case, no password is needed as authentication is done with the keys.

If you only need to clone and pull from the repo (and not push to it), you could set up Access keys on the Google instance that provide read-only access.

You can see instructions here, depending on the OS of the Google instance:

* https://support.atlassian.com/bitbucket-cloud/docs/set-up-repository-access-keys-on-linux/
* https://support.atlassian.com/bitbucket-cloud/docs/set-up-repository-access-keys-on-windows/
* https://support.atlassian.com/bitbucket-cloud/docs/set-up-repository-access-keys-on-macos/

The docs have instructions to generate an SSH key pair (you need to do that on the Google instance) and then add the public key to the Bitbucket repo's Access keys (from Repository settings > Access keys). You could then use an SSH URL in the git clone command, and git fetch would also use the same SSH URL.

If you want to use HTTPS, is there a credential manager set up for Git on the Google instance?

You can figure that out if you connect to the Google instance as the user $SSH_USER (outside Pipelines) and run the command

git config --list

Is there any line with the text "credential.helper"? If so, which one is used?

Are there any saved credentials for bitbucket.org in that credential manager?

Kind regards,
Theodora