We´ve migrated an existing Repository to make use of GIT-LFS.
The Repository is hosted at BitBucket Cloud while the bamboo server (latest version) is self-hosted behind a secured connection (vpn).
We´re using the Bamboo Source Code Checkout Task.
Bamboo uses a cached repository (/build-dir/_git-repositories-cache/...) to fasten the build process.
The initial clone to this cached repository path seems to work, all lfs files are downloaded successfully (as seen in the bamboo build log).
We tested this multiple times by flushing the cached repository contents and restart the process.
As far as we understand the bamboo source code checkout process the next step is to checkout the sources in the bamboo build dir
(in our case /build-dir/FSCWEB-FSCWEBSYMFONY-SS). The source code checkout (using a git remote pointing to file://CACHED_REPO)
works but after downloading the LFS files, which seems to work:
>> Git LFS: (31838 of 31838 files) 2.40 GB / 2.40 GB
the logs shows errors:
>> error: cannot feed the input to external filter true
>> error: external filter true failed
After the sourcode checkout task is finished, the pure git files are availale.
But the lfs files are missing (not existing in the filesystem) or have a incorrect filesize (0 or are very low and incorrect size).
This leads to multiple problems:
- git status or git lfs status shows many modified or deleted files (all from the lfs storage)
- the build process does not work as the files are required by following build stages.
Our current solution is to remove all files from the bamboo build dir and do a manually git-lfs clone from the bitbucket cloud repo url.
This downloads all files successfully (correct size, no git working dir changes after the clone). Then we start the bamboo build plan.
We´ve not tested to manipulate lfs files yet.
The git-lfs clone from the bitbucket repository works without problems at multiple developer machines, so the problem seems to be located at bamboo.
Our main question:
- How to fix the described problem?
- how does the bamboo git-lfs checkout task work in detail?
- Is a git-lfs clone used?
- Why does bamboo change the git remote url to file://CACHED_DIR during build instead of using the original Bitbucket Cloud url?
thank you for your answer!
There´s no .gitconfig in the bamboo build dir, here´s the output from the global config:
$ git config --global --list
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
As discribed earlier the clone/checkout works without any problems at many developer machines so the problem seems to be located at the bamboo server.
We did a lot of research and I still think the problem is located at the way how bamboo handles the cached repository.
The git origin and lfs endpoint points to the cached repository with git-lfs client "file://"
$ git lfs env
git-lfs/2.2.1 (GitHub; linux amd64; go 1.8.3)
git version 1.9.1
But as described at the github git-lfs website git-lfs has no "file://" support (github.com/git-lfs/git-lfs/wiki/Tutorial)
Note: only http:///https:// endpoints are supported by the git-lfs client. There is currently no direct ssh:// or file://. This means you can NOT have a local file lfs repo like you can for git.
Bamboo lfs support is enabled (as described here: https://www.atlassian.com/blog/bamboo/new-features-bamboo-5-15). Can you explain in detail how the bamboo git-lfs support looks like. As long as bamboo changes the origin and lfs endpoint to "file://" it won´t work.
Thank you for your help
Could I ask if there has been an answer to this question? If so, could someone post it here?
I notice the question is over a month old. This seems like a true bug in Bamboo's Git LFS support. I am trying to do the same thing with a large asset-heavy db and though, like you, the inital clone is good (and long!) subsequent pulls just pull pointer files.
we´ve got no answer from the atlassian support. They changed their policy and with a starter package you get no personal support any longer. Only this forum an as you see no help ;-(
I disabled the bamboo sourcecode checkout task and implemented a script based git checkout.
Over the next several weeks we'll be sharing some of our Getting Started guides here in the community. Throughout this series of posts, we'd love to hear from customers and non-customers ab...
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG
You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs