Repository not shrinking after migration to LFS. What do?

[Matt Dittbenner]

Main problem:

I pushed a repository completely converted to LFS back to bitbucket, but the repository remains large.

Backstory:

Tests

After many tests and backups, I was successfully able to convert an existing repository to LFS. My initial tests were creating a mirror from a local repository, and using git lfs migrate to convert it. Locally, I was experiencing an issue where the objects/ STAYED the same size, but LFS files were ALSO created, effectively doubling the repository size. I resolved this issue when I cloned a mirror from bitbucket. There may have been a rogue branch locally that retained these files.

Now that the local repository was fully converted (lfs/ is big, objects/ is small) I pushed it to a NEW repository in bitbucket. All was well.

Script

So, I worked on creating a script that would MIGRATE an existing repository. I have successfully executed the script locally, and the bare mirror looks correct with much of the storage space migrated from objects to lfs. However, I am experiencing issues when I attempt to push it up to bitbucket remote.

• The first time, it seemed to push successfully, but no LFS files were uploaded.
  • On an investigation of the bitbucket history, the files I looked for showed they were in LFS, but the files didn't seem to exist.
  • I manually did git lfs push --all origin and they all showed up.
  • The settings of the repository showed the correct amount of LFS space was now used
  • The repository size did NOT go down.
• I've tried several incantations of git push in an attempt to resolve this with no luck.
• I've waited 12 hrs hoping that it might be a git gc issue on the bitbucket end.

Full Question

The actual intent here is to reduce the size of many repositories by migrating to LFS. My understanding is that git push --mirror --force should REWRITE history and replace everything.

I can't figure out why the git history is still large on bitbucket while locally, it is not.

• What should I look for?
• How do I resolve this?

[Mikael Sandberg]

It sounds like git lfs migrate did not clean up this history to remove your LFS files. When I converted a repository to use Git LFS I used BFG Repo-Cleaner, and it did a great job shrinking the repository down in size. It has been awhile since I did it, so cannot say by how much, I would have to spin up my test environment to check that.

[Mikael Sandberg]

You can find Atlassian's instructions here, Use BFG to migrate a repo to Git LFS

[Matt Dittbenner]

Firstly, THANK YOU for the reply!

I initially tried to use BFG, and it failed in many ways in my windows environment. After some research, many sources seemed to recommend the git lfs migrate instead. One reason is that the BFG tool spread .gitattributes all over the tree (wherever it finds files that match) rather than maintaining a single one in the root. I also wasn't able to get BFG to work and found others experiencing the same issues.

Ultimately, it's true there are several tools that work. The important part of this question is that by all indications, I was successful locally. I can't figure out why it's still large on remote and need assistance identifying and resolving the issue.

For example: take a look at this.

/c/workspace_jboss/migrate $ du -s -h project.backup/objects
38M project.backup/objects

/c/workspace_jboss/migrate $ du -s -h project.MIGRATING/{objects,lfs}
1.7M project.MIGRATING/objects
39M project.MIGRATING/lfs

As you can see, the local repository history HAS in fact been converted. After running git gc (with appropriate options), the objects folder dropped in size. However, the issue I'm having is that the remote does NOT drop in size when I push.

[H.Tupath]

I'm having the same issue having followed this howto:
https://notiz.dev/blog/migrate-git-repo-to-git-lfs

Looked all good, but now the Repo in Bitbucket is not smaller AND I have used 3GB of LFS space!

I don't want to use BFG because I don't understand what it is doing and I'm not comfortable with that.