Splitting a huge repository in to multiple smaller ones.

[Allen Huffman]

Last year, I brought over a Perforce repository in to Bitbucket. The tools I used managed to keep the history and tags, it seems. I then used a Sourcetree option that set them up with develop/feature/master branches.

I want to split this big repositor up in to many smaller ones. For example, if I had something like:

Firmware
    Arduino
    MSP430
    ESP8266
    ESP32
    PiZero

...and I want each of those Firmware sub-folders to be a new repository, is there an easy way to do this? Initial tests look like it's possible, but it has the entire history of all files which makes me think all the files are still in there.

I'd keep the original legacy repository backed up in case I ever needed to go back to a commit which pushed things to multiple projects, but in this example, I'd like the Arduino repository to only have folders from Arduino down, and history of those files.

GIT seems tricky since if I had a commit that pushed updated files to Arduino, PiZero, ESP32, etc. at the same time, I don't know what that would do. I guess best-case would be the commit would be there, but the files that were outside of Arduino would just not exist in that repository, and be unlinked from the commit?

Thanks for tips. I've read dozens of tips and Q&A answers with various approaches, but none seems to be exactly what I was looking for.

[Mikael Sandberg]

Hi @Allen Huffman,

Welcome to Atlassian Community!

Have a look at this KB, it outlines the steps that you need to do to refactor your repository into smaller ones. Basically what you do is create a new repository, clone the existing one into it, remove the link to the original one, remove everything that you do not want in the new repo, link the new repo to the new repo in Bitbucket and then push everything back to it.

[Blake Peavy]

Hi Mikael,

Would you have any suggestions for repositories that have commit history that touches multiple files in directories that are all being split off, maintaining the commit history? 

I believe the best way would be going on a file by file base, correct?

[Allen Huffman]

I appreciate your assistance.

I am checking this tutorial now. The command-line provided to get the splitpractice repository would not work for me  -- "git@bitbucket.org: Permission denied (publickey)."

git clone git@bitbucket.org:tutorials/splitpractice.git freshrepo

By going to the sitepractice repository web page, I could copy the clone command there (which has my username in it) and put it in freshrepo and obtain it, so perhaps that was the intent (username# versus git@).

But, at the filter-branch step, it also failed with "fatal: bad revision 'rm'":

git filter-branch --index-filter 'git rm --cached -r lildir lildir2' -- --all

I am currently doing this on a Windows 11 machine.

Is there an obvious thing I missed?

[Mikael Sandberg]

Are you running this from the git bash command line or Windows cmd? If you are doing it from cmd then you need to replace all ' with ", because Windows only recognize ".

[Mikael Sandberg]

@Blake Peavy The way I did it in the past when working with Git was to start fresh and ignore the history. You could still keep the original repository around if there is a need to see the history, but in 99% of the cases no one ever needed to see the history.

[Allen Huffman]

@Mikael Sandberg DOS quotes helped get further. At the end I was able to link it to my  username@bitbucket.org/username/freshrepo.git with no errors, but then at "git push origin main" it reported:

error: src refspec main does not match any
error: failed to push some refs to 'https://bitbucket.org/allenhuffmancellencor/freshrepo.git'

...and nothing pushed back up. But much closer. Once I get this working, I'm going in to the sample repo and adding some tags and branches and such to see what happens to those when they are moved over this way.

[Mikael Sandberg]

Have a look at this blog post, it could be that the main branch is missing in the repo.