Git Submodule Authentication via https

Stephan Krusche June 24, 2013

Hi,

I'd like to build an application in Bamboo which is stored in a git repository that depends on a git submodule. Both repositories are stored in Stash, required https authentication via username and paswsword and our build user has access rights.

If I configure the git repository in Bamboo's build plan, there is the option to enable submodules. Unfortunately the submodule clone does not succeed because of an authentication exception:

Submodule 'framework' (https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git) registered for path 'framework'
24-Jun-2013 17:29:01    Cloning into 'framework'...
24-Jun-2013 17:29:01    fatal: Authentication failed
24-Jun-2013 17:29:01    Clone of 'https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git' into submodule path 'framework' failed
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:132)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:88)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:191)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:176)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.execute(VcsCheckoutTask.java:109)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:188)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.task.TaskExecutorImpl.executePreparationTasks(TaskExecutorImpl.java:69)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:69)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:206)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111)
24-Jun-2013 17:29:01            at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52)
24-Jun-2013 17:29:01            at java.lang.Thread.run(Thread.java:680)
24-Jun-2013 17:29:01    Caused by: com.atlassian.bamboo.repository.RepositoryException: Checkout to revision 4d58bfaf90508c753a0538bf06f8d8fc777f5d43 has failed.command /Applications/Xcode.app/Contents/Developer/usr/bin/git submodule update --init --recursive failed with code 1. Working directory was [/opt/bamboo-agent-home/xml-data/build-dir/GAME-GAME-CHECKOUTBUILD]., stderr:

I assume the problem is, that for the submodule, Bamboo does not automtically take the same authentication type (we use Username/Password) as for the normal repository. In our submodule file we do not include the username so that all participating developers are able to clone the git repository including the submodule with SourceTree (it works without any problems with SourceTree)

[submodule "framework"]
    path = framework
    url = https://repobruegge.in.tum.de:8443/scm/FRAMEWORK/framework.git


It is also not possible to specify any authentication options for the submodule.

Is there any solution or workaround for my problem?


Thanks
Stephan

8 answers

1 accepted

3 votes
Answer accepted
Ivan Maduro
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
June 24, 2013

Hi Stephan,

You're running into this bug BAM-11369, indeed checking out the Submodule isn't working because the authentication isn't actually going through the main Bamboo instance as a proxy. To workaround this you will need to set up authentication via SSH Keys.

You will need to generate an SSH key, store it on the agent (under the Bamboo agents home/.ssh dir)

And set up the appropriate .ssh/config for it. This will allow Git to authenticate against the submodules server and check out your Submodule correctly.

Cheers,

Ivan.

Matthias Linhuber July 25, 2013

Is there no other solution than SSH keys?

Jack Davis July 28, 2013

I'm a little confused at this answer because apparently HTTPS works alright according to the referenced bug. However this answer suggests that HTTPS doesn't work at all and that you've got to use SSH.

Help us out here Atlassian, does HTTPS work? Or do we have to use SSH?

Przemek Bruski
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 28, 2013

HTTP should work if you embed username/password into the submodule url, which the OP doesn't want to do. If you can't embed credentials, your only option is SSH.

Jack Davis July 30, 2013

If you can't use SSH in ondemand, does that mean you cannot use this workaround?

Przemek Bruski
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 30, 2013

On OnDemand, you can only use the HTTP with embedded credentials (I think they should work).

Jack Davis July 31, 2013
but if you don't want to embed credentials you're out of luck? Would a netrc file work?
Przemek Bruski
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 31, 2013

I think it should work. For submodules, any standard credentials provisioning method supported natively by Git will work.

Jack Davis July 31, 2013

Should I place it in the bamboo user's home directory? (Windows)

Jack Davis July 31, 2013

I just tried putting it in /Users/Bamboo and it didn't authenticate in bamboo.

However when I put it in the administrator's account and clone from gitbash it works.

I'm assuming the %HOME% directory of the Bamboo user is different, do you know what it is?

Jack Davis July 31, 2013

Apologies, the /Users/Bamboo directory does work. I am now able to authenticate over HTTPS without embedding credentials.

1 vote
John Sposato July 25, 2013

Here's what we did:

1. Setup a user (bamboouser in our case) and gave Observer rights to projects with submodules

2. Added this user/password to the Source Repositories section of the plan setup

3. In our .gitmodules, used the https: Stash link instead of ssh:, and ran git submodule sync

Ran the build and the submodules were successfully checked out.

0 votes
Jerry Morrow January 19, 2017

I've been successfully clone git repos without manual authentication using the ssh key feature.  That's been good.  I've had an issue where the submodule update command does not use the ssh key, and I still need to type username/password.  This is a problem for scripted builds (either shell script, buildbot, etc.).

Has anyone else had this issue?  I've already changed my .gitmodules such that the submodule is identified by ssh, not https.  I thought that would be the golden ticket, but it wasn't.

I don't want to throw in the towel. smile

0 votes
Todd Morrison October 27, 2014

Atlasssian Team,

Please note John Sposato's response above allows submodules to be fetched.

Please add this to your site's project documention for git support on Bamboo.

Thanks!

Todd

 

Here's what we did:
1. Setup a user (bamboouser in our case) and gave Observer rights to projects with submodules
2. Added this user/password to the Source Repositories section of the plan setup
3. In our .gitmodules, used the https: Stash link instead of ssh:, and ran git submodule sync
 
Ran the build and the submodules were successfully checked out.
0 votes
Caleb Tote March 24, 2014

For the record, I tried embedding my credentials into my https connection string, and received the same error. Our repo is hardcoded to not allow ssh (or rather, only allow https) - so we're pretty stuck.

0 votes
Kevin Dixon October 3, 2013

This is a terrible issue, but in the mean time, I have had success using

http://gitcredentialstore.codeplex.com/

on our windows build machines to pre-cache credentials for the submodules.

0 votes
John Sposato July 25, 2013

Done, but I still need the information for the workaround. Anyone?

0 votes
John Sposato July 25, 2013

Ivan,

What needs to be in the config for SSH? I am attempting to get this to work. We have a project that uses submodules from GitHub (which are fine) and Stash (get second auth prompt and it eventually fails).

Is there not a document on Atlassian's site to explain how to do this?

Jack Davis July 25, 2013

John please vote on this bug BAM-11369 so we can get it fixed.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events