It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bitbucket Pipelines and git submodules

Gediminas Šedbaras Jul 20, 2016

Hello!

How can I make Pipelines work when my repo contains git submodules that need to be initialized before the build is started?

6 answers

9 votes
Jacob Ideskog Aug 31, 2017

This seems fixed now with the SSH keys available in pipelines.

 

1. Create an SSH keypair for the repo that runs the pipeline that needs to init a submodule

 

2. Add the public key to the submodule repo.

 

3. add `git submodule update --init` to your pipeline script

Santiago GL Dec 27, 2017

Works. Thanks.

Gijs Wobben Mar 12, 2018

Works perfectly! Thanks!

tri.nm May 16, 2018

It works. Thanks !

As a note, you should remove 

$USERNAME:$PASSWORD

from your git path in your git commands inside your pipeline script.

Because only the ssh key is enough for authentication. 

Like Jacob Ideskog likes this
5 votes
Sebastian Cole Atlassian Team Jul 20, 2016

you'll need to reconfigure the git remote to pull from Bitbucket. Pipelines uses an O-auth token to do the initial cloning which is then destroyed immediately afterwards.

First you'll want to create an app-password for the user who's going to access the submodules (App passwords are here in Bitbucket Cloud | Bitbucket Blog). This will ensure you can work with 2fa enabled on the account.

Then you'll need to provide the username and app-password as Bitbucket Pipelines variables (https://confluence.atlassian.com/display/BITBUCKET/Environment+variables+in+Bitbucket+Pipelines)

assuming you've named them USERNAME and PASSWORD, add this step to the start of your pipeline:

git remote set-url origin https://$USERNAME:$PASSWORD@bitbucket.org/team/repo.git

followed by initialising your submodules

git submodule update --init
Lee Tze Ken Aug 19, 2016

Hi I have tried 20+ times but until the `git submodule update --init` step, it always said:

+ git submodule update --init
Submodule 'dev/scripts/api' (git@bitbucket.org:team/services.git) registered for path 'dev/scripts/api'
Cloning into 'dev/scripts/api'...
Warning: Permanently added 'bitbucket.org' (RSA) to the list of known hosts.

Permission denied (publickey).

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@bitbucket.org:team/services.git' into submodule path 'dev/scripts/api' failed

 

Anyone has experienced this?

 

Michael Dahlke Apr 03, 2017

Did you solve this issue?

 

I'm running into the same problem :(

Nissim Lal Pradhan May 17, 2017

Please note that, the environment variable $USERNAME mentioned by Sebastian should be account username instead of the app password username. The password should be the password from one of the app passwords you've created in account settings.

Like Karim Heraud likes this
Mike Medved Feb 20, 2019

I've added SSH keys from the pipeline repo to my submodule bitbucket repos, but I continue to get this error when trying to do a submodule update

fatal: could not read Username for: No such device or address

Seems clearly related, help?

Mike Medved Feb 20, 2019

note that bitbucket.org was up in the fatal line, but Atlassian communities kept barfing on it, even in code or block quote.  Sup with that?

5 votes
Sunny Wong Apr 18, 2017

It surely is a "massive missing feature in BB Pipelines". After 9 months passing by, is this fixed now? or is it still a manual submodule update?

4 votes
Romanas Sonkinas Jul 25, 2016

That isn't enough.

If your submodules point at an ssh Bitbucket url (git@bitbucket.org:org/repo.git), you need to change the url of the submodule as well. I did it like this:

 

git config --file=.gitmodules submodule.$SUBMODULENAME.url https://$USERNAME:$PASSWORD@bitbucket.org/org/repo.git

Then it's the standard git submodule update --init.

This feels like a massive missing feature in BB Pipelines and I hope it gets resolved in future updates!

1 vote
Soeren Balko Aug 14, 2017

Atlassian, this needs urgent fixing! I thought the advantage of using Bitbucket pipelines over some external CI system was the better integration with Bitbucket. Turns out that's not the case. Sad!

0 votes
Brian Maissy Jul 09, 2018

For anyone else who found this thead and thinks this is a serious oversight, you can vote on the issue here https://bitbucket.org/site/master/issues/12853/private-submodule-support

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

3,330 views 3 22
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you