Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,455,449
Community Members
 
Community Events
175
Community Groups

Bitbucket Pipelines and git submodules

Hello!

How can I make Pipelines work when my repo contains git submodules that need to be initialized before the build is started?

8 answers

This seems fixed now with the SSH keys available in pipelines.

 

1. Create an SSH keypair for the repo that runs the pipeline that needs to init a submodule

 

2. Add the public key to the submodule repo.

 

3. add `git submodule update --init` to your pipeline script

Works. Thanks.

Like mcw likes this

Works perfectly! Thanks!

Like mcw likes this

It works. Thanks !

As a note, you should remove 

$USERNAME:$PASSWORD

from your git path in your git commands inside your pipeline script.

Because only the ssh key is enough for authentication. 

Like # people like this

This worked for me as well. 

To be clear, add the public key as an Access Key in the `submodule` repo.

Like Byron Appelt likes this

It surely is a "massive missing feature in BB Pipelines". After 9 months passing by, is this fixed now? or is it still a manual submodule update?

5 votes
SebC Atlassian Team Jul 20, 2016

you'll need to reconfigure the git remote to pull from Bitbucket. Pipelines uses an O-auth token to do the initial cloning which is then destroyed immediately afterwards.

First you'll want to create an app-password for the user who's going to access the submodules (App passwords are here in Bitbucket Cloud | Bitbucket Blog). This will ensure you can work with 2fa enabled on the account.

Then you'll need to provide the username and app-password as Bitbucket Pipelines variables (https://confluence.atlassian.com/display/BITBUCKET/Environment+variables+in+Bitbucket+Pipelines)

assuming you've named them USERNAME and PASSWORD, add this step to the start of your pipeline:

git remote set-url origin https://$USERNAME:$PASSWORD@bitbucket.org/team/repo.git

followed by initialising your submodules

git submodule update --init

Hi I have tried 20+ times but until the `git submodule update --init` step, it always said:

+ git submodule update --init
Submodule 'dev/scripts/api' (git@bitbucket.org:team/services.git) registered for path 'dev/scripts/api'
Cloning into 'dev/scripts/api'...
Warning: Permanently added 'bitbucket.org' (RSA) to the list of known hosts.

Permission denied (publickey).

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@bitbucket.org:team/services.git' into submodule path 'dev/scripts/api' failed

 

Anyone has experienced this?

 

Did you solve this issue?

 

I'm running into the same problem :(

Please note that, the environment variable $USERNAME mentioned by Sebastian should be account username instead of the app password username. The password should be the password from one of the app passwords you've created in account settings.

Like Karim Heraud likes this

I've added SSH keys from the pipeline repo to my submodule bitbucket repos, but I continue to get this error when trying to do a submodule update

fatal: could not read Username for: No such device or address

Seems clearly related, help?

Like Vinay Kumar likes this

note that bitbucket.org was up in the fatal line, but Atlassian communities kept barfing on it, even in code or block quote.  Sup with that?

just spent a whole afternoon looking into this... the answer provided by Jacob Ideskog does work. However, one important point is to make sure the module url in`.gitmodules` is something like git@bitbucket.org:XXXX but not using https:. Only when it's set toSSH it will pick up the public key you generated and added to the submodule's repo. Otherwise, it will complain about either the username or password cannot be found.

Hope this helps 

That isn't enough.

If your submodules point at an ssh Bitbucket url (git@bitbucket.org:org/repo.git), you need to change the url of the submodule as well. I did it like this:

 

git config --file=.gitmodules submodule.$SUBMODULENAME.url https://$USERNAME:$PASSWORD@bitbucket.org/org/repo.git

Then it's the standard git submodule update --init.

This feels like a massive missing feature in BB Pipelines and I hope it gets resolved in future updates!

I followed the steps shown on this: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Managed to get the git submodule checkout working fine.

Only issue is, we use IP Whitelisting for our Bitbucket Account. When that is enabled, the submodule checkout fails! 

Warning: Permanently added the RSA host key for IP address '18.205.93.0' to the list of known hosts.

To access this repository, an admin must whitelist your IP.

fatal: Could not read from remote repository.

I tried whitelisting this IP and a bunch of others that it uses, but no luck! 

Anyone has managed to get a workaround for this? 

Same problem here. Did you have any luck fixing the problem? I'll raise a ticket otherwise.

Atlassian, this needs urgent fixing! I thought the advantage of using Bitbucket pipelines over some external CI system was the better integration with Bitbucket. Turns out that's not the case. Sad!

Hi
I would also like to use a submodule in my pipeline.
The submodule has been initilaised, but unfortunately

git submodule update --init

fails with the error:

fatal: destination path '<pathToSubmodule>' already exists and is not an empty directory,

because during the build setup the node_modules were already loaded from the cache.
Is there a solution for this?

use git submodule update --init --force

Hi @Valerii Kulyk_ Mindk_com
thanks for you answer, but this is not working. I still get the same error.

The --force option only causes the submodule to be cloned with git checkout --force, but does not fix the error that the submodule's folder is not empty.

For anyone else who found this thead and thinks this is a serious oversight, you can vote on the issue here https://bitbucket.org/site/master/issues/12853/private-submodule-support

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events