Not able push ( or create pointers in .git/lfs/objects folder ) binary files in when using Bitbucket

[Amit Jain]

It is a long post, as I tried to include as much detail as possible.

I am trying to use GIT LFS in Bitbucket, but for some strange reason, I am not able to push any of the binary files to LFS

Files considered for the test :

jar, so, exe, tar.gz, zip, .aff, .dic . .aff and .dic files are of ~30 MB per file size and dictionary files ( non-binary files)

When I am using git lfs, it will always throw error at the time of push for the above files, except .aff and .dic files. It does not matter whether I want to track them for lfs or not. But if I do not use lfs, I can push all the files to bitbucket (git).

While using lfs , it never creates the pointers in .git/lfs/objects folders for the binary, which creates problem at the time of push.

Here are the test steps I follow :

• Create a git repo ( using Bitbucket UI )
• Enable LFS in settings ( using Bitbucket UI )

• Clone that repo to local

  git clone ssh://git@git..com/~/gitlfs_test.git

• Initialize lfs

  cd /home//gitlfs_test/;git lfs install

• Initialize lfs tracking

  git lfs track "*/**/*.jar" ;
  git lfs track "*/**/*.tar.gz" ;
  git lfs track "*/**/*.zip" ;
  git lfs track "*/**/*.so" ;
  git lfs track "*/**/*.aff" ;
  git lfs track "*/**/*.dic" ;
  git lfs track "*/**/*.exe" ;

• git add .gitattributes

• git commit –m “commit 1”

• git push origin master( push successful )

• Copy the above mentioned files from other sources to the folder:

  bash-4.1$ pwd /home/<user>/gitlfs_test 

• mkdir jar so exe dict mkdir –p zipped/tar/; mkdir zipped/zip;

  for file in `find /home/backup -name '*.so'`; do cp $file so/;done
  for file in `find /home/backup -name '*.tar*'`; do cp $file zipped/tar/;done
  for file in `find /home/backup -name '*.zip*'`; do cp $file zipped/zip/;done
  for file in `find /home/backup -name '*.aff*'`; do cp $file dict/;done
  for file in `find /home/backup -name '*.dic*'`; do cp $file dict/;done`

• git add –all

• git commit –m “commit 2”

• git push origin master -- > push fails for jar files localhost{<user>}/home/<user>/gitlfs_test=> git push origin master b060a124dc0793068e83bb6facf6fcedf3ee57fd657764a07e8622 does not exist in .git/lfs/objects. Tried jar/SybDriver.jar, which matches b060a124dc0793068e83bb6facf6fcedf3ee57fd657764a07e8622. error: failed to push some refs to 'ssh://git@git.<myurl>.com/~<user>/gitlfs_test.git'

• I added step 6,7 and 8 as in many blogs, it was suggested to push .gitattributes files before we push the actual files.

• I face errors even if I push the .gitattributes files along with other files, that is, even if I skip step 6-8.

• In step 10 if I add only “.aff” and “.dic” files ( for which pointers are created in ./git/lfs/objects folder ) , the push is successful.

• Instead of git clone, if I just create a folder and carry on the activities until git commit, and the add the repository using git remote, I get the same result.

• Now this is the most shocking test result : I carried out the same test with only jar, aff and dic files, and tracked only aff and dic files

  • If I add ( git add ) only aff and dic files, while using lfs , git push works fine.
  • If I add all the 3 type of files using git add, I get the same error as step 12, even if I do not track jar. In short, by any means, I cannot push the binary files to git.

• Not only that, the push fails even when I try to create a lfs repo only for the jar files, but do not track any files. Please see the result as follows: 

localhost{<user>}/home/<user>=> git clone ssh://git@git.<myurl>.com/~<user>/gitlfs_test2.git

Cloning into 'gitlfs_test2'... warning: You appear to have cloned an empty repository.

localhost{<user>}/home/<user>=> cd gitlfs_test2 localhost{<user>}/home/<user>/gitlfs_test2=> ls -la

total 12

drwxrwxr-x 3 <user> users 4096 Jul 19 13:07 ./

drwxrwxr-x 18 build users 4096 Jul 19 13:06 ../

drwxrwxr-x 7 <user> users 4096 Jul 19 13:07 .git/ localhost{<user>}/home/<user>/gitlfs_test2=> bash bash-4.1$ mkdir jar

bash-4.1$ for file in `find ../home/backup -name '*.jar'`; 
do cp $file jar/;done

bash-4.1$ exit 

exit 

localhost{<user>}/home/<user>/gitlfs_test2=> ls -lrt .git 

total 32 

drwxrwxr-x 2 <user> users 4096 Jul 19 13:07 branches/ 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:07 hooks/ 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:07 info/ 
drwxrwxr-x 4 <user> users 4096 Jul 19 13:07 refs/ 
-rw-rw-r-- 1 <user> users 73 Jul 19 13:07 description 
-rw-rw-r-- 1 <user> users 23 Jul 19 13:07 HEAD 
drwxrwxr-x 4 <user> users 4096 Jul 19 13:07 objects/ 
-rw-rw-r-- 1 <user> users 273 Jul 19 13:07 config 

localhost{<user>}/home/<user>/gitlfs_test2=> git lfs install 
Updated pre-push hook. 
Git LFS initialized. 

localhost{<user>}/home/<user>/gitlfs_test2=> ls -lrt .git 

total 36 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:07 branches/ 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:07 info/ 
drwxrwxr-x 4 <user> users 4096 Jul 19 13:07 refs/ 
-rw-rw-r-- 1 <user> users 73 Jul 19 13:07 description 
-rw-rw-r-- 1 <user> users 23 Jul 19 13:07 HEAD 
drwxrwxr-x 4 <user> users 4096 Jul 19 13:07 objects/ 
-rw-rw-r-- 1 <user> users 273 Jul 19 13:07 config 
drwxr-xr-x 4 <user> users 4096 Jul 19 13:08 lfs/ 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:08 hooks/ 

localhost{<user>}/home/<user>/gitlfs_test2=> ls -lrt 
total 4 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:08 jar/ 

localhost{<user>}/home/<user>/gitlfs_test2=> ls -la 
total 16 
drwxrwxr-x 4 <user> users 4096 Jul 19 13:07 ./ 
drwxrwxr-x 18 build users 4096 Jul 19 13:06 ../ 
drwxrwxr-x 8 <user> users 4096 Jul 19 13:08 .git/ 
drwxrwxr-x 2 <user> users 4096 Jul 19 13:08 jar/ 
localhost{<user>}/home/<user>/gitlfs_test2=> git add --all 
localhost{<user>}/home/<user>/gitlfs_test2=> git commit -m "commit 1" 
[master (root-commit) ce7010b] commit 1 14 files changed, 42 insertions(+) 
create mode 100644 jar/SybDriver.jar 
create mode 100644 jar/aaaa.jar 
create mode 100644 jar/bbbb.jar 
create mode 100644 jar/cccc.jar 
create mode 100644 jar/dddd.jar 
create mode 100644 jar/eeee.jar 
create mode 100644 jar/ffff.jar 
create mode 100644 jar/gggg.jar 
create mode 100644 jar/hhhh.jar 
create mode 100644 jar/iiii.jar 
create mode 100644 jar/jjjj.jar 
create mode 100755 jar/kkkk.jar 
create mode 100644 jar/llll.jar 
create mode 100644 jar/mmmm.jar 
localhost{<user>}/home/<user>/gitlfs_test2=> git push origin master 
b060a124dc0793068e83bb6facf6fcedf3ee57fd657764a07e8622 does not exist in .git/lfs/objects. 
Tried jar/SybDriver.jar, which matches b060a124dc0793068e83bb6facf6fcedf3ee57fd657764a07e8622. 
error: failed to push some refs to 'ssh://git@git.<myurl>.com/~<user>/gitlfs_test2.git'

Please note, if I create a non-binary file and change the extention to .jar, git push ( when lfs is initialized ) succeeds. That is, pointer is created in .git/lfs/objects folder if I track .jar files.

Also, size is not an issue. the .aff and .dic files are in MBs, where as the jar files are in KBs.

The issue happens only if I use a binary file and try to push it to bitbucket as LFS. 

Here is the content of .gitattibutes file :

localhost{<user>}/home/<user>/gitlfs_test2=> cat .gitattributes
*/**/*.jar filter=lfs diff=lfs merge=lfs -text
*/**/*.tar.gz filter=lfs diff=lfs merge=lfs -text
*/**/*.zip filter=lfs diff=lfs merge=lfs -text
*/**/*.so filter=lfs diff=lfs merge=lfs -text
*/**/*.aff filter=lfs diff=lfs merge=lfs -text
*/**/*.dic filter=lfs diff=lfs merge=lfs -text
*/**/*.exe filter=lfs diff=lfs merge=lfs -text

I did not use github, as that is not available in my office.

Any help is appreciated !

Thanks so much !