Pushing Large Files and getting Memory Errors - Help Edited

Hi There,

We are starting a game development studios and have about 1T of data backups that we would like to add to a repository. 

We currently have a Linux (Ubuntu) BitBucker Server running on a virtual machine - this seems to be working fine.

When I try to push commits of large sizes within source tree, we are running into memory errors - any advice on how to fix this?

For example, we will be working on high-end engines like Crytek, and they recommend checking the entire engine into source integrity which 3-6 Gigs (thousands of files) alone not including some of the large source files that we need to deal with for artist and other creatives.  

Thank you.

Best regards,
Denis

 

2 answers

2 accepted

1 vote

I would take a look at using Git LFS, developed by GitHub, Atlassian and others to handle large files. We use is to store large video files and database binaries in our repositories.

0 vote

Thank you for the quick response.  I have LFS enabled for bitbucket - perhaps it is not enabled properly with source tree. 

Mikael Sandberg Community Champion Dec 20, 2017

The memory errors you see sounds like the large files have not been added to Git LFS, but still exist in the repository. Also, if the files were in the repository before you setup Git LFS, did you clean up the history of the repository to move those files out of it?

All new to this - sorry.  This is a brand new setup - just trying to set up initial repositories from a clean install/directory.  I did set LFS check mark in bitbucket - using Sourcetree as the interface (where something might not be set properly - all fresh installs (of everything! :).  

Basically, there are very large backup directories with engines, source art, video etc - in various directories.  We want to make this all accessible to the team.

Mikael Sandberg Community Champion Dec 20, 2017

Okay, so you probably need to add those directories to Git LFS, this command will do that:

git lfs track "<path to backup directory>/"

This will add the directory and all files under it to Git LFS. In the directories that have files added to Git LFS you will see a .gitattributes file that keeps track on files added to Git LFS. The tutorial I linked to above is really good at describing what you need to do to set everything up and how to use it.

Great thank you - is there a way to do this in SourceTree? 

 

We tried the command line and it says git "does not know LFS" command.  This is a fresh sourcetree with bitbucket install.

Not if you want to add whole directories. In Sourcetree you only have the option to add file types after you initialize the repository the repository (Repository > Git LFS > Initialize Repository).

Ok thank you - we are trying to initialize the LSF directory now through command lines.  The Initialize Repository option is currently greyed out.

Hi Mikael,

Thank you for your help.  With your help, we have finally understood how to do this correctly.  The interface with SourceTree messed us up for a bit, but we found a process by jumping between command line commands and SourceTree visuals to correctly set the right files (by extensions) to LFS and set the regular text files to use GIT.  We have now successfully pushed 5-6 Gigs worth of files and testing a 70 directory to bitbucket, and it seems to be working.  Next, we have a 1 T directors :p.  

Thanks again!

Best regards,
Denis

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Posted Jun 06, 2018 in Bitbucket

Do you use Bitbucket Cloud and Jira Cloud? If so, let us know!

Hi Community, I'm Julia and I'm on the Jira Software Cloud marketing team!  We're looking for companies or teams using Bitbucket Cloud and Jira Software Cloud. If your team fits the t...

155 views 6 3
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you