Does Bit-Booster always require double the space when rebase/squash/amend is enabled?

Steven Whitman January 3, 2017

Form the setting page there is the statement with the rebase/squash/amend feature enabled, Bit-Booster maintains a git clone directly on the server's disk, potentially doubling the amount of disk space required by the Bitbucket Server instance.

Since this feature is global enabled does that mean that I'll suddenly need twice the disk storage or is this the most pessimistic case?  Are the clones done using hardlinks? Also when a repository is deleted is the clone deleted?

2 answers

0 votes
Steven Whitman January 4, 2017

Thanks for the detailed answer.  I guess it would also be interesting to understand how the clone's disk usage increases over time as more pull requests are performed.  For example if I rebase a very large commit, I would assume that it will forever exist in the clone and the source repo.

G__Sylvie_Davies__bit-booster_com_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
January 5, 2017

I've put some logic in place to make sure the bit-booster clone hardly grows (beyond the minimum it needs to checkout the commit for the rebase, e.g., the working copy).  The clone doesn't even bother storing refs (branch labels and tag labels) after its initial clone operation. It never does a "git fetch" or a "git pull".  In the case you've mentioned it would construct the commit for the huge rebase, but once the commit was successfully pushed it deletes the commit on its side.

It does maintain a single copy of the working copy at all times, and usually this is where most of the disk usage is happening.  

0 votes
G__Sylvie_Davies__bit-booster_com_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
January 4, 2017

I'm still running experiments to determine disk usage.  Instead of using hardlinks it uses "git clone --shared" to save space, since in many installs the "bit-booster" clone (under $BITBUCKET_HOME/caches/bbClones) will be on a different file system (and so hardlinks won't work).  In particular I needed to make sure my clone was not on the NFS mount used by Bitbucket Data Center clusters.

This worst case would be a repository with only a single commit.  In that case I suspect the disk increase would be close to a 100% of the repo's current size.

Repos will a lot of commits tend to require less disk (relative to the repo's total size).  I've tested disk utiliization with a 1.1GB repository with 15,000 commits going back 5 years. The working area reported usage of 230MB, and yet the system report (df -h) reported a decrease of only 150MB. I suspect the add-on typically increases per-repo disk usage by about 25% of the repo's current size.

I plan to run more experiments, and I will post them here.

If you ever need to free up disk it is safe to run "rm -rf $BITBUCKET_HOME/caches/bbClones" at any time.  Any time a repository's pull-request screen is accessed it will reclone to that location in the background (if Bit-Booster's rebase feature is enabled).  Repositories that never use pull-requests, or are never accessed, will not use more disk with this feature enabled.

 

At this time stale clones are not deleted, but I plan to include a stale clone reaper in the January 9th 2017 release I'm currently planning.

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events