For context, I work for an architectural visualisation studio. Myself and a number of 3D artists/motion graphics people and another coder work to produce videos and apps. The apps tend to involve fairly hefty 3D models/textures, and this has fairly frequently led to repositories becoming unnecessarily bloated.
Recently, we've had one go to 1.9 Gb, which is Too Close. Having done a variety of cleanup/size-reduction operations locally,
git count-objects -vH
tells me that my local repository is 1.44 Gb.
However, even when I force push to the Bitbucket remote, the settings page there tells me the Bitbucket repo is still 1.9 Gb. Am I missing something here? The local repo has the complete history in 1.44Gb, and I'd been under the impression that the --force flag for push would cause the remote's history to match the local. Although I've used git for years for personal projects, I've not frequently had cause to use remotes for those, so this is a bit beyond my experience. Is there something else I should be doing to make the remote carry out the cleanup/size-reduction measures?
(For the record: I'm aware that even 1.4 is too damn big for a sensible code repository. Being able to diff ASCII encoded FBX files is useful however, and I'm in the process of establishing a workflow to split the textures out rather than having to keep them in the repo.)
The force push did "orphan" those objects on our side, but as you suspected, that is not necessarily immediately updated on our side.
Calculating repo sizes can be relatively expensive and so we only do it once every so often, depending on a series of heuristics.
The other thing that is relevant here is that when rebasing/filter-branch'ing large objects out of your repo, that does not immediately delete them from disk. They are merely orphaned and become "garbage". You will still have to run
git gc to reclaim disk space.
In turn, even
git gc does not immediately reclaim space, as under normal circumstances garbage objects aren't deleted until they have reached a certain age (2 weeks default) and even then there are heuristics at play. You should be able to see this if you compare the output of
git count-objects -vH with
du -sh .git/ on the local repo where you performed the cleanup.
Your repo size on our end will eventually drop, but without manual intervention it's hard to give a reliable time for that.
We realize that this can cause problems for very large repos that are really close to the 2GB hard limit (as you are) and we are in the process of making changes to prune and reclaim disk space more aggressively for repos like your own.
Thanks for confirming. The cleanup steps I followed for local cleanup were "git reflog expire --expire=now --all", followed by a "git gc --aggressive --prune=now", which will have killed the garbage on this end. We've had to actually reset a few commits (and then force push them up), so I'm guessing those'll generate significant quantities of orphaned junk in the BitBucket repo, even if we've cleaned it out locally. Shouldn't repeat itself, since I've gotten the other coder up to speed on what shouldn't be going into commits and when they should be pushed up, but there was quite a lot of bulk data in there. We did actually end up having to do the "Open a new empty repo and use that" solution, since the original went over 2 Gb yesterday. The original's now deleted, but your answer's a better general solution, so will do that moving forward. Cheers.
My other suspicion is that the BitBucket repository settings page just doesn't show the size in real time, and only updates occasionally. It is at least still working for the moment, so we'll keep working on it for the next couple days. Since the local repos are much smaller, hopefully the BitBucket remote'll catch on that it's smaller too. If I'm wrong and it does hit the 2 Gb then we'll make up a new one.
On October 21st, 2020 we hosted a webinar titled, Step Up You DevOps Game with 4 Key Integrations for Jira and Bitbucket. We had a great showing and high engagement, but that meant th...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events