Delete large file from git repo and push changes to bitbucket

[iliavlad]

Hi,

I have the git project hosted on bitbucket.org.

In project settings I see " Size 1007.0 MB ".

Screenshot 1:

In my local git repo I did the following (I changed names of files,branches and others to 111,222,etc in the paste):

$ ls -l .git/objects/pack/
-r--r--r-- 1 322596  pack-924...a2.idx
-r--r--r-- 1 1051815075  pack-924...6a2.pack // omg

$ join -o "1.1 1.2 2.3" <(git rev-list --objects --all | sort) <(git verify-pack -v .git/objects/pack/*.idx | sort -k3 -n | tail -5 | sort) | sort -k3 -n
6fcc0164d4bd385574d32a1b96f91e121c3febb5 111.js 3056957
aed8b44ab488d5f1353c9a490d5160680e05e3e5 222.xml 3558522
6e12939fe172a45a8810c2e8515a99ee482f0d27 111.xml 3562020
f3ab19982ef60286859f5f721fd7600cca1125a1 111.sql 8611705
25600adf42936e6f7c4c8ff7c34d0fe00be2e9b9 backup.zip 1087243066 // got it

// delete
$ git filter-branch -f --tree-filter 'rm -f backup.zip' HEAD
Rewrite 750c98d7869942080b239d34fb89dc9cbd030cf4 (746/757) (60 seconds passed, remaining 0 predicted)
WARNING: Ref 'refs/heads/master' is unchanged

// delete
$ git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch backup.zip' --prune-empty --tag-name-filter cat -- --all
Rewrite 1a6a5d35a06ba47ea8e4ec1c542a36bdd1d0124d (664/990) (25 seconds passed, remaining 12 predicted) rm 'backup.zip'
Rewrite 246b70283d7d1703d6f0144fe0946ddf82273e65 (664/990) (25 seconds passed, remaining 12 predicted) rm 'backup.zip'
...
Ref 'refs/remotes/origin/111' was rewritten
Ref 'refs/remotes/origin/222' was rewritten
WARNING: Ref 'refs/remotes/origin/333' is unchanged

$ git for-each-ref --format='delete %(refname)' refs/original | git update-ref --stdin
$ git reflog expire --expire=now --all
$ git gc --aggressive --prune=now
$ git prune; git repack -ad

$ ls -l .git/objects/pack/
-r--r--r-- 1 319404 pack-674...ea3.idx
-r--r--r-- 1 10288157 pack-674...ea3.pack // i like it
 

In my local repo I have the pack file of 10 Mb instead of 1Gb. And now I want to push changes to bitbucket. But I don't know how to do it :-(

$ git push -f
...
To bitbucket.org:.../....git
+ 45e3baf...750c98d master -> master (forced update)

 I pushed with -f with no effect.

At "branch permission" I did "Allow rewriting branch history".

Now repo has 1Gb size at bitbucket. How can I fix it?

Thanks.

[DPKJ · Accepted solution]

To remove large file from repository, you can use following commands,

$ git filter-branch --tree-filter 'rm path/to/your/bigfile' HEAD
$ git push origin master --force

And if you repository size doesn't reduce, as per this documentation https://confluence.atlassian.com/bitbucket/reduce-repository-size-321848262.html You can create new repo on bitbucket, update remote to point to new location and push

[iliavlad]

I done "create new repo on bitbucket, update remote to point to new location and push".

Thanks.

[Daniil Penkin]

Hello @iliavlad,

Generally there can be two options basically: either you haven't deleted that large file from the history completely OR Git garbage collection hasn't run on the server side yet (it's an expensive operation and is not run on every repository update).

I was able to find the repository in question and triggered GC manually. However, the repo size didn't decrease much (only by few megabytes). It looks like there's still some ref pointing at a commit where that huge file still exists. Can you double check that? I can see a warning in the log you posted – maybe that's related?

You can also perform a test by pushing the same repo to a new place, like @DPKJ suggested – just to try out how much space does a fresh repository take.

Hope this helped.

Cheers,
Daniil