our repo has got several submodules. In order to reduce pipeline runtimes We'd like to cache the submodule data and fetch only the changes.
Unfortunately the pipepline crashes if the submodules are cached because the initialiazation doesn't work proberly.
right now the following command gets executed
git submodule update --init
Does anyone of you know if submodule caching is possible and if so how to handle that.
The solution is to cache the .git/modules directory, and not the individual submodule directories.
- git submodule update --recursive --init
The "git module" command fails b/c the directory exists (from cache) but there is no matching directory inside .git/modules.
Once I had it working, our "git submodule" step went from 35 seconds to 6 seconds.
At the moment there is no support for git submodule caching in Pipelines. You could setup different pipelines in different repositories so that every repository clones its own changes. If that does not work for your particular use case, I recommend you open a ticket in our site master issue tracker (https://bitbucket.org/site/master/issues) with your use case so that we can track more customer uses cases and votes related to it.
You should be able to cache your submodules the same way all other cache works.
In your bitbucket-pipelines.yml add a new definition of the submodule folder you would like to cache and then reference that cache in your build step.
Here is a small example. I have split the step of creating cache separate to the actual test as the cache is not built if the test fails. Though you can setup submodules and run tests in the same step.
pipelines: default: - step:
name: Downloading Submodules caches: - submodules script: - git submodule update --init definitions: caches: submodules: [submodule dir]
The pipeline looks something similiar to this:
pipelines: default: - step: caches: - submodule1
- submodule2 script:
- alter submodule1 url with pipeline user credentials
- alter submodule2 url like the first one - "git submodule update --init"
- run a gulp build
- deploy definitions: caches: submodule1: [submodule1 dir]
submodule2: [submodule2 dir]
Without the caching the pipeline works fine. But with caching enabled the following error occurs:
fatal: Not a git repository: ../.git/modules/submodule1
Unable to find current revision in submodule path 'submodule1'
Hello! My name is Mark Askew and I am a Premier Support Engineer for products Bitbucket Server/Data Center, Fisheye & Crucible. Today, I want to bring the discussion that Jennifer, Matt, and ...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs