Binary reject control not working correctly

[Devtools_Barclays]

We have implemented the binary reject control and it works fine if we just push the code for the first time. BUt if we amend the commit to remove the file and push to BBS, it fails to stop the push . Also we saw the repo size before the push and after it , there was a size increase in the repo.Please let us know if you require any other info from our side.Thanks

Steps To Reproduce1. Note the size of the Git repo in the BBS user interface (Settings -> retrieve size)

 2. Commit a binary file to a Git repo 

3.Push to BBS, you will get a rejection message 

4.Amend your commit to remove the file from the commit (git rm binFile.exe; git commit --amend) 

5.Push the commit to BBS - it will pass Note the size of the Git repo in the user interface (Settings -> retrieve size) - it will be much larger 

6.Mirror clone the repository locally to a new location (git clone --mirror ...)

 7.Perform some analysis on the repo git rev-list --objects --all | sort -k 2 > allfileshas.txt git gc && git verify-pack -v objects/pack/pack-*.idx | egrep "^\w+ blob\W+[0-9]+ [0-9]+ [0-9]+\w" | awk '{print $1,$3}' | sort -k 2 -n -r > bigobjectsall.txt for SHA in `cut -f 1 -d\ < bigobjectsall.txt`; do echo $(grep ^$SHA bigobjectsall.txt) $(grep $SHA allfileshas.txt) >> bigtosmallall.txt done; Check the file bigtosmallall.txt You should see a line referring to the file without a path, eg. $ less bigtosmallall.txt 2491b7745f89d0cf89bcb601a850563e90e4a0a8 14296728 e96534fb27b68192f27f985d3879e173ec77adb8 8792 e96534fb27b68192f27f985d3879e173ec77adb .idea/uiDesigner.xml 

8.Proof that the binary file has actually been pushed to BBS and bypassed the control (although the file is not visible in the repo, it is still carried around in the commit history)

Script used:

// Exempt repository IDs
if (repository.id in [10337]) {
return false;
}

// Reject the following filetypes, except if the file is gradle-wrapper.jar
return pathsMatchExcludingDeletes('regex : (?i:.*
.(dll|exe|zip|tar|gz|pdb|msi)$)')

com.onresolve.jira.stash.groovyrunner

[adammarkham]

From what I can see this is an issue with all pre-receive hooks. Bitbucket will collect the objects that are unreferenced when it does a full garbage collection, although the file does not appear in the commit history on Bitbucket.

This is an issue with all pre-receive hooks and is not limited to ScriptRunner.

Please see: https://confluence.atlassian.com/bitbucket/maintaining-a-git-repository-321848291.html

You can read more about git's garbage collector, and ways to manually invoke it at https://git-scm.com/docs/git-gc. Take note of the gc.reflogExpire and gc.reflogExpireUnreachable options in the Configuration section if you want to expire old objects more quickly.

You may want to take this up with Atlassian Support but don't mention ScriptRunner as they will just send you back to us. You'll probably see the same behaviour when you have no hooks and then commit and push a large file and then later remove it, the size of the repository will not reduce to reflect that until garbage collection kicks in.

I'm not saying this is a bug this is how Bitbucket and Git work, just Atlassian can provide more support on how to do the garbage collection etc.