Problem with git submodules in Bamboo coupled with local cache

[yogesh_devi]

Dear Bamboo experts,

I have a Bamboo build plan that uses git submodules . The build is failing with a message as follows . The problem seems to be that bamboo agent is trying to clone submodules from local cache and clone is failing because that operation needs a password . See log below 

Already on 'dev/mainrepo'
simple	27-May-2019 12:32:10	/usr/bin/git remote set-url origin ssh://d5fcbb24-a632-4e2e-948f-489ca955c411@127.0.0.2:41106/project/mainrepo.git
simple	27-May-2019 12:32:10	/usr/bin/git submodule update --init --recursive
simple	27-May-2019 12:32:11	Cloning into 'submodule1'...
simple	27-May-2019 12:32:11	Warning: Permanently added '[127.0.0.2]:41106' (RSA) to the list of known hosts.
simple	27-May-2019 12:32:11	Permission denied (password).
simple	27-May-2019 12:32:11	fatal: Could not read from remote repository.

If I try to do the same operation from agent command line  I see following message which tells me that the passwordless auth is NOT set-up for my local cache. 

git submodule update --init --recursive
Cloning into '/home/bamboo/bamboo-agent-home/xml-data/build-dir/BUILDNAME/submodule1'...
12ff7ca1-452e-4836-920e-8d44bc@127.0.0.1's password: 

Please note that my bamboo agent account ssh key is already uploaded to bitbucket and if I try to checkout and do submodule update from the git repository on bitbucket everything works fine ( without prompting me for password). It asks me for password though when I run the build from bamboo because then it tries to clone from local cache on agent 

Does someone know the way I can set-up passwordless auth for bamboo local cache. 

Alternatively I am okay to turn off the caching.

I tried to turn off caching from Bamboo gui by un-checking the checkbox for caching repositories. However Bamboo seems to cache regardless of the checkbox if submodules are involved 

I read https://jira.atlassian.com/browse/BAM-18265 -- still trying to figure what I could do though 

[yogesh_devi · Accepted solution]

This problem remains unresolved till date - but i devised following work around 

#  I created a task that does this sequence to explicitly update submodules  

git remote remove origin
git remote add origin ${bamboo.repository.git.repositoryUrl}
git remote set-url origin ${bamboo.repository.git.repositoryUrl}
git remote -v
/usr/bin/git submodule deinit --force .
/usr/bin/git submodule update --init --recursive

[Kaspars Laizans]

Had same issue.
1. Generate ssh keys on bamboo server under bamboo user (not atlassian software), add them to your ssh-keytool
$ ssh-keygen -m PEM -t rsa -C bamboo@myserver

$ eval $(ssh-agent)

$ ssh-add ~/.ssh/id_rsa

2. copy private key to (GUI) Bamboo/Config/Shared Credentials

3. Add pubkey to Bitbucket GUI to all the submodule projects that host submodule repositories

4. Create GIT NOT BITBUCKET linked repo, as mentioned by Daniel above. This forces Bamboo to use system git which supports submodules, in stead of bundled whatever gutted git version they have. Select shared SSH credentials you entered above

[Daniel Santos]

Hi @yogesh_devi

Thank you for sharing a detailed problem description. That is very helpful.

Can you confirm you are using the Git repository type and not Bitbucket one to configure your source repository in Bamboo?

The Git option should be able to use the submodules with your local SSH keys if the option [x]Use submodules is checked in the repository advanced options configuration.

[yogesh_devi]

hi @Daniel Santos  Thank for responding. I am using "Bitbucket" repository .  How to use "Git repository" as you mention ?

[Daniel Santos]

You will keep using the Bitbucket source repository, but in Bamboo side, you will use a different method for linking the repository. Instead of using the Bitbucket linking you will use the Git one as shown below:

:
More details on how to configure can be found here: Git - Atlassian Documentation.
Let me know if you have other questions in the meantime.

[ianburrowssa]

Hi,

Did this end up working? I have created a git repo as described above. Used an SSH URl and used my private key, tested the connection which was successful but submodule cloning still does not work.

[Anil Unnikrishnan]

I am facing the same issue as well.