$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
fatal: Out of memory, malloc failed (tried to allocate 4168062027 bytes)
fatal: sha1 file '<stdout>' write error: Broken pipe
error: remote unpack failed: unpack-objects abnormal exit
I am trying to push *.sql which is around 4Gb.
Can you help me get over this issue? Is there any limitation on the file size in the repository?
Hi @Mangesh Chougule ,
Thank you for taking the time and provide us with a bit more of context on the problem you are facing.
The suggestion will be the same as previous mentioned, however, I will be providing you with more explanation as to why you should follow the suggested.
You are attempting on pushing a file (3.8Gb in size) to a repository. If you are not using Git LFS, whenever user or CI tool that will pull from this repository will subsequently request that 3.8Gb SQL file. This will cause a great load on your Bitbucket DataCenter when "git-upload-pack" gets issued on the server and all of this could be avoid if you push larger files using Git LFS.
As pointed out ( https://www.atlassian.com/git/tutorials/git-lfs ) Git clients will be able to fetch/pull from your repository and only when needed they would be issuing a Git LFS command to retrieve those larger files.
On top of that, if not only your Git client but also the Git version installed and used by Bitbucket DataCenter is 2.18.0 or above, you can benefit from SSH Protocol v2 (this feature became available on Bitbucket v5.13 or above - Bitbucket Server 5.13 release notes).
It is best practises and recommended using SSH over HTTP(S) protocol when working with larger repositories, because the SSH protocol is much more reliable and it is meant for this purpose. Please, review Using SSH keys to secure Git operations documentation as it provides you with necessary information as:
Just as a reminder, the size of your repository should be given by the instructions provided on Git Repository Size from the Command Line. The command "du -hs" does not actually give the size of a repository.
If SSH is not an option, then you should be reviewing:
Kind regards,
Rafael
Hi Rafel,
Thanks for the response.
Im trying to push 3.8 GB file to a git server from my windows mahine using git bash.
1. git version 2.23.0.windows.1
2. Windows - 64 bit
3.Ram = 16gb
4. $ du -hs >> 4.7G
5. Data centre- GIT server is hosted.
6. What is the protocol you are using (HTTP(S) or SSH)?
Ans: HTTPS
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi @Mangesh Chougule ,
Based on the error provided, we are unable to confirm if you "remote" to where you are running the git push command is defined as HTTP(S) or SSH protocol. For larger files, you should be considering using:
If you really need to have this file available to your repository and not to a Git LFS (Large File Storage) then you could play with ".gitconfig" settings related to pack (https://git-scm.com/docs/git-config):
...
[pack]
threads = 1
deltaCacheSize = 128m
packSizeLimit = 128m
windowMemory = 128m
[core]
packedGitLimit = 128m
packedGitWindowSize = 128m
So we can have a bit more of context:
Perhaps you should consider reading about Scaling Bitbucket Server in which provides examples about Memory and CPU consumption based the way you work with your repository. Furthermore, a comparison between HTTP(S) vs SSH protocol.
Kind regards,
Rafael
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Online forums and learning are now in one easy-to-use experience.
By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.