Git Submodule Authentication via https

Hi,

I'd like to build an application in Bamboo which is stored in a git repository that depends on a git submodule. Both repositories are stored in Stash, required https authentication via username and paswsword and our build user has access rights.

If I configure the git repository in Bamboo's build plan, there is the option to enable submodules. Unfortunately the submodule clone does not succeed because of an authentication exception:

Submodule 'framework' (https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git) registered for path 'framework'
24-Jun-2013 17:29:01    Cloning into 'framework'...
24-Jun-2013 17:29:01    fatal: Authentication failed
24-Jun-2013 17:29:01    Clone of 'https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git' into submodule path 'framework' failed
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:132)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:88)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:191)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:176)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.execute(VcsCheckoutTask.java:109)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:188)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.task.TaskExecutorImpl.executePreparationTasks(TaskExecutorImpl.java:69)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:69)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:206)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52)
24-Jun-2013 17:29:01            at java.lang.Thread.run(Thread.java:680)
24-Jun-2013 17:29:01    Caused by: com.atlassian.bamboo.repository.RepositoryException: Checkout to revision 4d58bfaf90508c753a0538bf06f8d8fc777f5d43 has failed.command /Applications/Xcode.app/Contents/Developer/usr/bin/git submodule update --init --recursive failed with code 1. Working directory was [/opt/bamboo-agent-home/xml-data/build-dir/GAME-GAME-CHECKOUTBUILD]., stderr:

I assume the problem is, that for the submodule, Bamboo does not automtically take the same authentication type (we use Username/Password) as for the normal repository. In our submodule file we do not include the username so that all participating developers are able to clone the git repository including the submodule with SourceTree (it works without any problems with SourceTree)

[submodule "framework"]
    path = framework
    url = https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git


It is also not possible to specify any authentication options for the submodule.

Is there any solution or workaround for my problem?


Thanks
Stephan

8 answers

1 accepted

3 votes
Accepted answer

Hi Stephan,

You're running into this bug BAM-11369, indeed checking out the Submodule isn't working because the authentication isn't actually going through the main Bamboo instance as a proxy. To workaround this you will need to set up authentication via SSH Keys.

You will need to generate an SSH key, store it on the agent (under the Bamboo agents home/.ssh dir)

And set up the appropriate .ssh/config for it. This will allow Git to authenticate against the submodules server and check out your Submodule correctly.

Cheers,

Ivan.

Is there no other solution than SSH keys?

I'm a little confused at this answer because apparently HTTPS works alright according to the referenced bug. However this answer suggests that HTTPS doesn't work at all and that you've got to use SSH.

Help us out here Atlassian, does HTTPS work? Or do we have to use SSH?

HTTP should work if you embed username/password into the submodule url, which the OP doesn't want to do. If you can't embed credentials, your only option is SSH.

If you can't use SSH in ondemand, does that mean you cannot use this workaround?

On OnDemand, you can only use the HTTP with embedded credentials (I think they should work).

but if you don't want to embed credentials you're out of luck? Would a netrc file work?

I think it should work. For submodules, any standard credentials provisioning method supported natively by Git will work.

Should I place it in the bamboo user's home directory? (Windows)

I just tried putting it in /Users/Bamboo and it didn't authenticate in bamboo.

However when I put it in the administrator's account and clone from gitbash it works.

I'm assuming the %HOME% directory of the Bamboo user is different, do you know what it is?

Apologies, the /Users/Bamboo directory does work. I am now able to authenticate over HTTPS without embedding credentials.

Here's what we did:

1. Setup a user (bamboouser in our case) and gave Observer rights to projects with submodules

2. Added this user/password to the Source Repositories section of the plan setup

3. In our .gitmodules, used the https: Stash link instead of ssh:, and ran git submodule sync

Ran the build and the submodules were successfully checked out.

Ivan,

What needs to be in the config for SSH? I am attempting to get this to work. We have a project that uses submodules from GitHub (which are fine) and Stash (get second auth prompt and it eventually fails).

Is there not a document on Atlassian's site to explain how to do this?

John please vote on this bug BAM-11369 so we can get it fixed.

Done, but I still need the information for the workaround. Anyone?

This is a terrible issue, but in the mean time, I have had success using

http://gitcredentialstore.codeplex.com/

on our windows build machines to pre-cache credentials for the submodules.

For the record, I tried embedding my credentials into my https connection string, and received the same error. Our repo is hardcoded to not allow ssh (or rather, only allow https) - so we're pretty stuck.

Atlasssian Team,

Please note John Sposato's response above allows submodules to be fetched.

Please add this to your site's project documention for git support on Bamboo.

Thanks!

Todd

 

Here's what we did:
1. Setup a user (bamboouser in our case) and gave Observer rights to projects with submodules
2. Added this user/password to the Source Repositories section of the plan setup
3. In our .gitmodules, used the https: Stash link instead of ssh:, and ran git submodule sync
 
Ran the build and the submodules were successfully checked out.

I've been successfully clone git repos without manual authentication using the ssh key feature.  That's been good.  I've had an issue where the submodule update command does not use the ssh key, and I still need to type username/password.  This is a problem for scripted builds (either shell script, buildbot, etc.).

Has anyone else had this issue?  I've already changed my .gitmodules such that the submodule is identified by ssh, not https.  I thought that would be the golden ticket, but it wasn't.

I don't want to throw in the towel. smile

Suggest an answer

Log in or Sign up to answer
Community showcase
Published 11 hours ago in Statuspage

194 years of downtime: looking back on incident data from 2018

Statuspage customers logged more than 194 years of collective incidents in 2018. That’s a whopping 87% increase from the  104 years logged in 2017 , and we aren’t even through December yet....

23 views 0 3
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you