LFS files marked as changed after clone in pipeline

Question

I've got repo with 4 binary files store in LFS and when my pipeline clones this repo and runs a `git status` or `git diff` command sometimes 1 or more of the binary files are reported as modified, however when I check the files (via sha256sum) I can find no discernible difference.  Which files are reported as modified is totally random, it can be all 4 files or just 1 or 2 or none of them.

So far this has only been observed with the pipeline build. The only other thing of note is that the pipeline is running as self hosted on my own hardware.

I've checked that the files are properly marked as binary and lfs in the .gitattributes file.

I'm really stumped on what is causing this, any help would be greatly appreciated.

Theodora Boudale · Answer

Hi Matt,

Thank you for the update, It's good to hear that you have resolved the issue.

I'm not sure if the version mismatch you mention was the issue. Considering that git lfs --version was not showing the version of Git LFS, I would assume that it might have been a bug with the Git LFS version that was previously installed in the Docker container.

Kind regards,
Theodora

Matt Schuckmann · Answer

Hi Theodora,

I think we've resolved the problem by installing the latest version of git and git-lfs in the docker container running the pipeline. The docker container is based on Ubuntu 18.04 with git and git lfs installed from apt packages so the versions were pretty old.  I went ahead and installed both from source and so far the problem has not come back.

It's still not clear to me why we only saw the problem when running from the pipeline. I suppose a version mismatch between the git that does the clone and the git that checks the status could cause the problem, is that possible.

Here is a thread from the git-lfs github that helped me 
https://github.com/git-lfs/git-lfs/issues/3787

Matt S.

Theodora Boudale · Answer

Hi Matt,

Thank you for your reply.

1. If you run the following commands on a machine of yours that has Git LFS installed

git clone --branch="feature/resolve-pipeline-dirty-issue" --depth 50 <REPO_URL>
git reset --hard 51ee15fd053bca2b24162e877f027d0720c7cced
git status

do you see files as modified in the output of git status or not? I'm asking this so we can narrow down whether this may be a repo-specific issue or not.

2. Have you always been tracking the files listed in .gitattributes with Git LFS in this repo, or were such files previously tracked by Git and you migrated them to Git LFS later (e.g. with BFG or git lfs migrate)?

Kind regards,
Theodora