Can't free up LFS storage

[Coleman Horton]

My LFS storage is full and I'm trying to free up space, but having issues.  The first thing I did was to update my .gitignore to ignore image and audio files, which should remove the vast majority of data in LFS, but I can't push because I still get the error:  "Your LFS push failed because you're out of file storage"

After that I manually deleted the biggest files from LFS and freed up over 300MB, but I'm still getting the same error despite only needing to push 257MB.

Why can I still not push despite having enough LFS storage?  How can I get the LFS storage to update according to my new .gitignore settings?

[Theodora Boudale]

Hi @Coleman Horton,

I checked the Bitbucket workspace associated with your community account email, and I see that the current LFS usage is 671.7 MB. I also synced the LFS usage between our systems in case there was a delay with syncing.

Could you please try pushing now and let me know how it goes?

In case you are still having issues, could you please try for testing purposes to make a smaller push (a push that includes less than the 257MB files of LFS) to see if that works?

Kind regards,
Theodora

[Coleman Horton]

Hi,

I tried pushing a single text file, but it still fails with Sourcetree apparently still trying to push 254 MB.

[Coleman Horton]

I also tried pushing a single file via command line and it hangs uploading LFS objects.  The problem seems to be that it's not removing LFS files from the remote repo.  How can I clear the remote LFS files?

[Theodora Boudale]

Hi @Coleman Horton,

I checked the LFS usage of your plan and it has been increased since yesterday, right now it's a little less than 1 GB.
Based on that and on what you report, it looks like some LFS files have been uploaded, but I cannot see if it's the ones that you previously deleted or new LFS files.

Could you please share some more details about your current LFS setup and your end goal, so I can better help you and give you guidance specific to your use case?

Specifically, it would be useful to know:

• What types of files are listed now in the .gitattributes file of your repo?
  
  
• What is your end goal?
  Do you want to completely remove from Git LFS and ignore ALL files of a specific extension, e.g. .jpg?
  Or, alternatively, continue to track e.g. .jpg files with Git LFS, but remove some of the .jpg files from Git LFS?

Kind regards,
Theodora

[Coleman Horton]

I want to remove and ignore all image and audio files from LFS.

I assumed that adding extensions to .gitignore would cause those files to be removed from the remote, but it seems like that might not always be the case if those are set to go to LFS in .gitattributes so I commented out all of the file types I'd like to remove in .gitattributes and tried pushing again.

I committed and pushed a single text file and Sourcetree sat all day pushing to source, but made no progress (my pushes usually take less than a minute).  I tried reinstalling and updating Sourcetree and I get the same behavior.

[Norbert Csupka]

Hi Coleman,

Let me step in. I'm Norbert and I'm also a Bitbucket Cloud Support Engineer, it's nice to meet with you!

I'd like to inform you that in case you add the extensions to .gitignore file, it won't remove those files from the LFS. Unfortunately we yet to have the feature to delete files from LFS in bulk. 

We have a feature request for this, which is:

BCLOUD-14585 - Add the ability to remove LFS files in bulk

Unfortunately the only way to delete those files are one-by-one via the GUI.

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support

[Coleman Horton]

Thanks for the reply.  So there isn't even a way to do this via command line?  There are thousands of files I need to remove - deleting them one by one isn't even close to a solution.

[Norbert Csupka]

Hi Coleman,

When I was searching internally I found a workaround:

Please run the following command to obtain the LFS oid:

git lfs ls-files -al | awk -F " " '{print $1}'

Once you have the list of OID, you can run the following shell application to delete the LFS files:

#!/bin/bash

# Get list of OID through pointer file in repo 
### git lfs ls-files --debug | grep oid | awk 'FNR >= 1 {print $3}'


file="<path to text file>"
workspaceid="workspace_id"
repo_slug="repo_slug"
credential="<username:app_password>"

i=0

while read line || [ -n "$line" ]
do 
i=$((i+1))
echo "File number : $i"
echo "LFS Object ID (OID) : $line" 
cmd=$(curl -s -w "\nHTTP Code: %{http_code}\n" --user $credential -X DELETE "https://api.bitbucket.org/internal/repositories/$workspaceid/$repo_slug/lfs/$line")
echo "Response : $cmd"
echo ""
done < $file

echo ""
echo "Total LFS Object ID in $file :" 
grep -c "" $file

Please let me know how it goes.

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support