How to remove contents from Git when hardware limit was reached ?

Dear support,

My repository is currently blocked as I reached the 2 GB limit. However, I cannot delete some files anymore, although I have followed the guide given by Bitbucket.

Unfortunately, I have to present my results in 2 days in a very important congress, and I need an access to my Bitbucket repository as early as possible.

Many thanks for your help and best regards,
Thibault Derrien

12 answers

This widget could not be displayed.

I cannot delete some files anymore

What exactly have you tried to do? This is important information for a good bug/problem report.

Particularly, have you tried pushing local changes? e.g., `git rm whatever`, then `git commit -m [why you did it]`, then `git push [remote_branch_name] [local_branch_name]`?

This widget could not be displayed.

1/ I have tried to remove files which I don't use and which take space. Then tried to commit and push the changes, but I get always the same answer from repository:

Total 12103 (delta 5275), reused 11562 (delta 4993)
remote: repository is in read only mode (over 2 GB size limit).
To git@bitbucket.org:tjyderrien/octopus.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@bitbucket.org:tjyderrien/octopus.git'

2/ Then I followed the step indicated into the guide for large repositories, i.e.:
I installed and used BFG repo cleaner. I did remove the files bigger than 100 MB (but nothing was found) and I also removed once again the annoying files (all the "aims.out" files which are not useful and take ~ 400 Mo).

I tried to push the changes, it tooks hours, and then was refused by the server.

I don't know what to do. Should I create a new repository to work?

All the best,
Thibault

This widget could not be displayed.

Hi Thibault,

When a repo exceeds 2GB, we only accept pushes that are clear deletions of commits. So if any new commit is added, we deny the push. It is for that exact reason that the guide to dealing with reducing the repository size starts with the section Remove the repository limitation, which recommends that you remove the last few commits from BB before running something like BFG.

I just ran an aggressive GC on your repo, which has brought down the size of your repo to 1.8 GB. Please push your post-BFG repo now (the push should work now that the repo size is lowered). Then, when the next GC is run on your repo, the size should reduce quite a bit. Also, rather than waiting for the next GC to run automatically, just ping me (reply to this comment) after you've pushed and I'll run a manual GC again.

This widget could not be displayed.

Hello Abhin,

Many thanks for your help! smile
I have pushed my changes, and enabled Git LFS to avoid bad situation again. However, it looks that the size of repository has super increased (~ 3 GB now). Can you run an aggressive gc ?

Thanks for your help,
Thibault

This widget could not be displayed.

I ran the aggressive GC. The size is now lower than 2GB.

This widget could not be displayed.

Great. I will continue the washing up.
If I add files to Git LFS, can it decrease the Git repository size also? Or other GC will be needed?

This widget could not be displayed.

Removing history in Git involves creating new commits, moving existing refs to point at them and pushing everything. That still means that the old commits are still in the history. They are removed when we run GC on the server. So further cleanups still require GC. Now usually, Git GC runs automatically on the servers, but it was just faster for me to trigger it manually for now.

Using LFS only means that any future large files will not be a part of the core repo history. The existing commits with references to existing blobs will still be a part of your repo. Using LFS will help the repository size from growing in the future.

I recommend watching https://www.youtube.com/watch?v=iJ3hXuEVKAc to understand LFS and how it works. It's also kinda fascinating.

This widget could not be displayed.

I use this repository from several places, including clusters. It looks that I did not succeed to synchronize them. Can I ask you to again make an aggressive gc ?

This widget could not be displayed.

Hello,

Finally, the limit of 2GB is not sufficient for me, then I have removed my large files from the repository and will manage with TAR and SCP. Can I ask you to run an aggressive git gc, again?

Many thanks,
Thibault

This widget could not be displayed.

Hi Thibault,

I ran the aggressive GC, but it only decreased the repo size marginally. Perhaps you need to move more files into LFS and clean up your history?

This widget could not be displayed.

Hi! Thanks for your action.

I think I would prefer to abandon Git LFS, and remove binary files from my repository.

But then it means that files are still taking too much space. Have to clean more. And then clean history, indeed.

I will tell when it's ready.

This widget could not be displayed.

Hi Abhin,

I ran into the same problem as Abhin and can't push the changes because my repo is over 2 gb. Could you also run a GC on mine?

Thank you!

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Aug 21, 2018 in Bitbucket

Branch Management with Bitbucket

As a project manager, I have discovered that different developers want to bring their previous branching method with them when they join the team. Some developers are used to performing individual wo...

1,306 views 8 11
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you