Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,456,500
Community Members
 
Community Events
176
Community Groups

Repository not shrinking after migration to LFS. What do?

Main problem:

I pushed a repository completely converted to LFS back to bitbucket, but the repository remains large.

Backstory:

Tests

After many tests and backups, I was successfully able to convert an existing repository to LFS. My initial tests were creating a mirror from a local repository, and using git lfs migrate to convert it. Locally, I was experiencing an issue where the objects/ STAYED the same size, but LFS files were ALSO created, effectively doubling the repository size. I resolved this issue when I cloned a mirror from bitbucket. There may have been a rogue branch locally that retained these files.

Now that the local repository was fully converted (lfs/ is big, objects/ is small) I pushed it to a NEW repository in bitbucket. All was well.

Script

So, I worked on creating a script that would MIGRATE an existing repository. I have successfully executed the script locally, and the bare mirror looks correct with much of the storage space migrated from objects to lfs. However, I am experiencing issues when I attempt to push it up to bitbucket remote.

  • The first time, it seemed to push successfully, but no LFS files were uploaded.
    • On an investigation of the bitbucket history, the files I looked for showed they were in LFS, but the files didn't seem to exist.
    • I manually did git lfs push --all origin and they all showed up.
    • The settings of the repository showed the correct amount of LFS space was now used
    • The repository size did NOT go down.
  • I've tried several incantations of git push in an attempt to resolve this with no luck.
  • I've waited 12 hrs hoping that it might be a git gc issue on the bitbucket end.

Full Question

The actual intent here is to reduce the size of many repositories by migrating to LFS. My understanding is that git push --mirror --force should REWRITE history and replace everything.

I can't figure out why the git history is still large on bitbucket while locally, it is not.

  • What should I look for?
  • How do I resolve this?

1 answer

0 votes
Mikael Sandberg Community Leader Oct 05, 2018

It sounds like git lfs migrate did not clean up this history to remove your LFS files. When I converted a repository to use Git LFS I used BFG Repo-Cleaner, and it did a great job shrinking the repository down in size. It has been awhile since I did it, so cannot say by how much, I would have to spin up my test environment to check that.

Mikael Sandberg Community Leader Oct 05, 2018

You can find Atlassian's instructions here, Use BFG to migrate a repo to Git LFS

Firstly, THANK YOU for the reply!

I initially tried to use BFG, and it failed in many ways in my windows environment. After some research, many sources seemed to recommend the git lfs migrate instead. One reason is that the BFG tool spread .gitattributes all over the tree (wherever it finds files that match) rather than maintaining a single one in the root. I also wasn't able to get BFG to work and found others experiencing the same issues.

Ultimately, it's true there are several tools that work. The important part of this question is that by all indications, I was successful locally. I can't figure out why it's still large on remote and need assistance identifying and resolving the issue.

For example: take a look at this.

/c/workspace_jboss/migrate $ du -s -h project.backup/objects
38M project.backup/objects

/c/workspace_jboss/migrate $ du -s -h project.MIGRATING/{objects,lfs}
1.7M project.MIGRATING/objects
39M project.MIGRATING/lfs

As you can see, the local repository history HAS in fact been converted. After running git gc (with appropriate options), the objects folder dropped in size. However, the issue I'm having is that the remote does NOT drop in size when I push.

I'm having the same issue having followed this howto:
https://notiz.dev/blog/migrate-git-repo-to-git-lfs

Looked all good, but now the Repo in Bitbucket is not smaller AND I have used 3GB of LFS space!

I don't want to use BFG because I don't understand what it is doing and I'm not comfortable with that.

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events