BFG for a Noob

jordan saia

This question is in reference to Atlassian Documentation: Reduce repository size

I have been asked by Bitbucket to reduce the size of my repo. 

I read the article and downloaded BFG. It makes no sense to me? Where do I input these commands?

Tried the Sourcetree console to no avail. 

Im very confused. Programmer is busy with other things so I, the artist, am attempting this. 

Can someone walk me through this?

MotionMaker

They need to be done at the command line on your computer. I am a Mac user so I use Terminal to get to the command line and GIT, which is what I am using, is installed so I can use it. 

It took me all day to clear one folder following the web page steps as there are many confusing steps and messages. After repeating a few times, and got the steps sifted down so it goes fairly quick. The longest time will be waiting for cloning  and removing if you have a lot of files, but certainly not more than an hour or three.

Bottom line they need to clarify this page is not for Noob even if you have some experience but for those who work at the command line daily. 

However I find that there are two traps I ran into so far. One is that BFG requires Java 7 but there is a link at the site for an earlier version, bfg-1.12.3.jar, that uses Java 6 as stated in the Bitbucket documentation. Use java -version at the command line to help with that process. 1.6 is version 6 and 1.7 is version 7.

Second I found Source Tree becomes confusing to use after you run the command line steps. It wants to put back all the files even if you have excluded them in .gitignore. This is because you need to clone the repo again and then add the files you now want to ignore back in.

My steps for over 1GB but under 2GB limit included

1. Clear any pending commits and do not add new files staging until you are complete and happy.
2. Get the repo url in Source Tree. Click Settings for the repo window, select it and then click edit. Something like git@bitbucket.org:YourUserName/your_repo_name.git
3. Make a backup of your local project folder with the files and repo. This will save your behind. Do not change it or delete it until time passes you are happy with final results. One way is just to make a copy of the folder but be sure all hidden files particular .git folder are copied.
4. Create an empty folder outside of your project folder. More in likely a sibling to the project folder.
5. Go into Terminal
6. At the command line: cd path_to_your_new_folder
7. At the command line: git clone --mirror git@bitbucket.org:YourUserName/your_repo_name.git
8. At the command line: One or more BGF commands to remove files you want to remove. In my case it was a giant folder and just one command line:
   java -jar bfg.jar --delete-folders my_folder --delete-files my_folder  --no-blob-protection  your_repo_name.git
9. At the command line: cd your_repo_name.git
10. At the command line: git reflog expire --expire=now --all && git gc --prune=now --aggressive
11. At the command line: git push 
12. Keep this folder because if you mistakenly add the deleted files back in to the remote repo on Bitbucket, you can come here just do the last line. 
13. Go into Bitbucket and not Source Tree. Open your repo and go to source. You should see the files you removed are now gone. The settings will not reflect the size reduction necessarily. That should happen when they run a clean operation on your repo.
14. In Bitbucket clone your repository to a new folder. Probably just make this another sibling to your other two project folders. Keep in mind it will be missing the files you removed earlier and all files you have been ignoring previously via the .gitignore file. You can use the SourceTree link in Bitbucket. If you have the SourceTree bookmark for the original and want to use that same name for this clone, you can rename it before you  proceed. Ultimately you will delete the original bookmark but not the local files.
15. Copy to the new repo folder the files from your backup folder that you removed from the repository in steps 9-10. For example if you removed a folder called images, you can put it back assuming you want to keep the files in the project.
16. Copy  to the new repo folder the files from your backup folder that you were excluding using .gitignore. This may be selective. In my case I occasionally use DW which litters the drive with dwsync.xml files. I ignore them and would not bother to copy them back in anyway. However I have a folder with local webserver logs that I ignore. Those I probably would copy back in.
17. Edit your .gitignore file in the newly cloned folder from step 14. You want to ignore the files you removed from the repo that were causing it to be too big. They may actually be appearing in the working copy window in SourceTree but will disappear once you save the .gitignore file. 
18. You can delete the SourceTree bookmark for the original project folder but keep the folder and files!
19. If you want to use the name project file folder for the new cloned folder, you need to rename the original folder (keep it because again it is your safety net), then rename the new clone folder to the original folder name, then you have to edit the settings for SourceTree to the old folder name. 
    
    

You also now need to find a good backup plan for those files outside the repo. For example you could use GoogleDrive, Cubby, Dropbox and other types of versioning services. 

The documentation page covers too many variations. They need separate pages for hitting the 2 GB limit and 1 GB limit. Then they need one for each of the repository types, git and mecurial. They need one for each of the tools.  

Second it would be worthwhile mentioning on the page that the garbage cleaning may take a while before you see the reset in the repo site in settings. 

Third the page does not attempt to be complete instructions. So there is a lot of fill in the blanks. 

Fourth, it may be a support reduction task to create a simple page for the user of SourceTree and include the steps for removing some large files or folders from start to end. 

Fifth, this is a potentially destructive process with possibility of permanent loss of the file you intend to remove from GIT but keep in your project due to misinterpretation of the many messages and steps that might go along with using SourceTree. Be careful and keep a good backup until you are clear of any doubts.