Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,293,318
Community Members
 
Community Events
165
Community Groups

Problem with git submodules in Bamboo coupled with local cache

Edited

Dear Bamboo experts,

I have a Bamboo build plan that uses git submodules . The build is failing with a message as follows . The problem seems to be that bamboo agent is trying to clone submodules from local cache and clone is failing because that operation needs a password . See log below 

Already on 'dev/mainrepo'
simple	27-May-2019 12:32:10	/usr/bin/git remote set-url origin ssh://d5fcbb24-a632-4e2e-948f-489ca955c411@127.0.0.2:41106/project/mainrepo.git
simple	27-May-2019 12:32:10	/usr/bin/git submodule update --init --recursive
simple	27-May-2019 12:32:11	Cloning into 'submodule1'...
simple	27-May-2019 12:32:11	Warning: Permanently added '[127.0.0.2]:41106' (RSA) to the list of known hosts.
simple	27-May-2019 12:32:11	Permission denied (password).
simple	27-May-2019 12:32:11	fatal: Could not read from remote repository.

 

If I try to do the same operation from agent command line  I see following message which tells me that the passwordless auth is NOT set-up for my local cache. 

 

git submodule update --init --recursive
Cloning into '/home/bamboo/bamboo-agent-home/xml-data/build-dir/BUILDNAME/submodule1'...
12ff7ca1-452e-4836-920e-8d44bc@127.0.0.1's password: 



Please note that my bamboo agent account ssh key is already uploaded to bitbucket and if I try to checkout and do submodule update from the git repository on bitbucket everything works fine ( without prompting me for password). It asks me for password though when I run the build from bamboo because then it tries to clone from local cache on agent 

Does someone know the way I can set-up passwordless auth for bamboo local cache. 

Alternatively I am okay to turn off the caching.

I tried to turn off caching from Bamboo gui by un-checking the checkbox for caching repositories. However Bamboo seems to cache regardless of the checkbox if submodules are involved 

 

I read https://jira.atlassian.com/browse/BAM-18265 -- still trying to figure what I could do though 

 

3 answers

1 accepted

0 votes
Answer accepted

This problem remains unresolved till date - but i devised following work around 

#  I created a task that does this sequence to explicitly update submodules  

git remote remove origin
git remote add origin ${bamboo.repository.git.repositoryUrl}
git remote set-url origin ${bamboo.repository.git.repositoryUrl}
git remote -v
/usr/bin/git submodule deinit --force .
/usr/bin/git submodule update --init --recursive

Had same issue.
1. Generate ssh keys on bamboo server under bamboo user (not atlassian software), add them to your ssh-keytool
$ ssh-keygen -m PEM -t rsa -C bamboo@myserver

$ eval $(ssh-agent)

$ ssh-add ~/.ssh/id_rsa

2. copy private key to (GUI) Bamboo/Config/Shared Credentials

3. Add pubkey to Bitbucket GUI to all the submodule projects that host submodule repositories

4. Create GIT NOT BITBUCKET linked repo, as mentioned by Daniel above. This forces Bamboo to use system git which supports submodules, in stead of bundled whatever gutted git version they have. Select shared SSH credentials you entered above

Hi @yogesh_devi

Thank you for sharing a detailed problem description. That is very helpful.

Can you confirm you are using the Git repository type and not Bitbucket one to configure your source repository in Bamboo?

The Git option should be able to use the submodules with your local SSH keys if the option [x]Use submodules is checked in the repository advanced options configuration.


hi @Daniel Santos  Thank for responding. I am using "Bitbucket" repository .  How to use "Git repository" as you mention ?

You will keep using the Bitbucket source repository, but in Bamboo side, you will use a different method for linking the repository. Instead of using the Bitbucket linking you will use the Git one as shown below:

Linked_repositories_-_Atlassian_Bamboo_-_Vivaldi.png:
More details on how to configure can be found here: Git - Atlassian Documentation.
Let me know if you have other questions in the meantime.

Like ianburrowssa likes this

Hi,

Did this end up working? I have created a git repo as described above. Used an SSH URl and used my private key, tested the connection which was successful but submodule cloning still does not work.

I am facing the same issue as well.

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bamboo

Bamboo Data Center on Kubernetes

Hi, If you are running self-managed environments and looking to adopt modern infrastructure, Bamboo Data Center can now be deployed in a Kubernetes cluster. By leveraging Kubernetes, you can easily...

909 views 3 8
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you