We´ve migrated an existing Repository to make use of GIT-LFS.
The Repository is hosted at BitBucket Cloud while the bamboo server (latest version) is self-hosted behind a secured connection (vpn).
We´re using the Bamboo Source Code Checkout Task.
Bamboo uses a cached repository (/build-dir/_git-repositories-cache/...) to fasten the build process.
The initial clone to this cached repository path seems to work, all lfs files are downloaded successfully (as seen in the bamboo build log).
We tested this multiple times by flushing the cached repository contents and restart the process.
As far as we understand the bamboo source code checkout process the next step is to checkout the sources in the bamboo build dir
(in our case /build-dir/FSCWEB-FSCWEBSYMFONY-SS). The source code checkout (using a git remote pointing to file://CACHED_REPO)
works but after downloading the LFS files, which seems to work:
>> Git LFS: (31838 of 31838 files) 2.40 GB / 2.40 GB
the logs shows errors:
>> error: cannot feed the input to external filter true
>> error: external filter true failed
After the sourcode checkout task is finished, the pure git files are availale.
But the lfs files are missing (not existing in the filesystem) or have a incorrect filesize (0 or are very low and incorrect size).
This leads to multiple problems:
- git status or git lfs status shows many modified or deleted files (all from the lfs storage)
- the build process does not work as the files are required by following build stages.
Our current solution is to remove all files from the bamboo build dir and do a manually git-lfs clone from the bitbucket cloud repo url.
This downloads all files successfully (correct size, no git working dir changes after the clone). Then we start the bamboo build plan.
We´ve not tested to manipulate lfs files yet.
The git-lfs clone from the bitbucket repository works without problems at multiple developer machines, so the problem seems to be located at bamboo.
Our main question:
- How to fix the described problem?
Further questions:
- how does the bamboo git-lfs checkout task work in detail?
- Is a git-lfs clone used?
- Why does bamboo change the git remote url to file://CACHED_DIR during build instead of using the original Bitbucket Cloud url?
Hi Tobias,
Could I ask if there has been an answer to this question? If so, could someone post it here?
I notice the question is over a month old. This seems like a true bug in Bamboo's Git LFS support. I am trying to do the same thing with a large asset-heavy db and though, like you, the inital clone is good (and long!) subsequent pulls just pull pointer files.
Thanks,
Bob
Hey Bob,
we´ve got no answer from the atlassian support. They changed their policy and with a starter package you get no personal support any longer. Only this forum an as you see no help ;-(
I disabled the bamboo sourcecode checkout task and implemented a script based git checkout.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
We have the same problems here...
I did a workaround with an additional script-task after the checkout.
In this script-task i call "git lfs checkout" explicitly in the repo folder.
I think this is a bit more time consuming than an builtin functionality, but it works... ^^
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi branden,
thank you for your answer!
There´s no .gitconfig in the bamboo build dir, here´s the output from the global config:
$ git config --global --list
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
As discribed earlier the clone/checkout works without any problems at many developer machines so the problem seems to be located at the bamboo server.
We did a lot of research and I still think the problem is located at the way how bamboo handles the cached repository.
The git origin and lfs endpoint points to the cached repository with git-lfs client "file://"
$ git lfs env
git-lfs/2.2.1 (GitHub; linux amd64; go 1.8.3)
git version 1.9.1
Endpoint=file:///mnt/bamboo_storage/bamboo/xml-data/build-dir/_git-repositories-cache/ ....
But as described at the github git-lfs website git-lfs has no "file://" support (github.com/git-lfs/git-lfs/wiki/Tutorial)
>
Note: only http:///https:// endpoints are supported by the git-lfs client. There is currently no direct ssh:// or file://. This means you can NOT have a local file lfs repo like you can for git.
>
Bamboo lfs support is enabled (as described here: https://www.atlassian.com/blog/bamboo/new-features-bamboo-5-15). Can you explain in detail how the bamboo git-lfs support looks like. As long as bamboo changes the origin and lfs endpoint to "file://" it won´t work.
Thank you for your help
ToBe
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Tobias,
It looks like a git lfs configuration error. Can you paste the contents of the .gitconfig file?
Cheers,
Branden
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.