`ssh git@bitbucket.org git-lfs-authenticate` hangs in git but not in shell

[Kiwi]

As the title says I'm having issues with lfs authenticate over ssh.

I'm on Windows 22H2 using openssh and gnupg gpg-agent as the ssh-agent. Don't be afraid of this and jump to this must be the issue becaues it works robustly for other ssh, git, and git-lfs combinations, and as the subject says works when tested directly.

C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]> ssh -V
OpenSSH_for_Windows_8.6p1, LibreSSL 3.4.3
C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]> git --version
13:41:47.434210 exec-cmd.c:243 trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:41:47.449919 git.c:462 trace: built-in: git version
git version 2.41.0.windows.1

My SSH keys (provided by GNUPG via gpg-agent) are working

• Bitbucket.org git
• github (including LFS)
• ssh in general

This is demonstrated by the usual ssh git test

C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]> ssh -T git@bitbucket.org
authenticated via ssh key.

You can use git to connect to Bitbucket. Shell access is disabled
C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]>

and the command in the subject working from the shell

C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]> ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git download
{"header":{"Authorization":"<token redacted>"},"href":"https://bitbucket.org/{}/{<redacted>}/info/lfs","expires_in":300}

When the same command is executed by git itself (I'm executing git in the same shell as the I executed the direct ssh command) it hangs.

C:\src\lfs-ssh-issue\lfs-auth-issue.bb [master ↑2]> git push
13:07:08.315130 exec-cmd.c:243 trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:07:08.315130 git.c:462 trace: built-in: git push
13:07:08.315130 run-command.c:661 trace: run_command: unset GIT_PREFIX; ssh git@bitbucket.org 'git-receive-pack '\''michael_thx/lfs-auth-ssh-issue.git'\'''
13:07:20.427673 run-command.c:661 trace: run_command: .git/hooks/pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git
13:07:20.506550 exec-cmd.c:243 trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:07:20.506550 git.c:748 trace: exec: git-lfs pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git
13:07:20.506550 run-command.c:661 trace: run_command: git-lfs pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git
13:07:20.530433 trace git-lfs: exec: git 'version'
13:07:20.562332 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote' '-v'
13:07:20.585132 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
13:07:20.600525 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
13:07:20.616556 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
13:07:20.633572 trace git-lfs: exec: uname
13:07:20.664992 trace git-lfs: exec: git 'config' '--includes' '-l'
13:07:20.680763 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
13:07:20.696471 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
13:07:20.712601 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
13:07:20.728453 trace git-lfs: attempting pure SSH protocol connection
13:07:20.728956 trace git-lfs: run_command: ssh -oControlMaster=auto -oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-1505662046\sock-%C git@bitbucket.org git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload
13:07:20.735051 trace git-lfs: exec: ssh '-oControlMaster=auto' '-oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-1505662046\sock-%C' 'git@bitbucket.org' 'git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload'
13:07:20.759777 trace git-lfs: pure SSH protocol connection failed: Unable to negotiate version with remote side (unable to read capabilities): EOF
13:07:20.760617 trace git-lfs: pre-push: refs/heads/master 7fa39d52acb51d4e34393b3b6afc559fb343da2d refs/heads/master 5bf600c8130a63a81f909f62d281108acecebc6d
13:07:20.760617 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'show-ref'
13:07:20.791762 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'ls-remote' '--heads' '--tags' '-q' 'origin'
13:07:25.979460 trace git-lfs: attempting pure SSH protocol connection
13:07:25.980075 trace git-lfs: run_command: ssh -oControlMaster=auto -oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-2198785769\sock-%C git@bitbucket.org git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload
13:07:25.987952 trace git-lfs: exec: ssh '-oControlMaster=auto' '-oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-2198785769\sock-%C' 'git@bitbucket.org' 'git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload'
13:07:26.010250 trace git-lfs: pure SSH protocol connection failed: Unable to negotiate version with remote side (unable to read capabilities): EOF
13:07:26.010250 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload
13:07:26.010250 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'
13:12:53.251588 trace git-lfs: filepathfilter: creating pattern ".git" of type gitignore
13:12:53.251588 trace git-lfs: filepathfilter: creating pattern "**/.git" of type gitignore
13:12:53.252124 trace git-lfs: filepathfilter: accepting "tmp"

Exiting because of "interrupt" signal.

The "interrupt" is me hitting Ctrl-C. I've let it wait overnight and after about 6 hours... the connection is reset

C:\src\lfs-ssh-issue\lfs-auth-ssh-issue [master ↑2]> git push

00:30:22.854519 exec-cmd.c:243          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin

00:30:22.854519 git.c:462               trace: built-in: git push

00:30:22.854519 run-command.c:661       trace: run_command: unset GIT_PREFIX; ssh git@bitbucket.org 'git-receive-pack '\''michael_thx/lfs-auth-ssh-issue.git'\'''

00:30:26.222016 run-command.c:661       trace: run_command: .git/hooks/pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git

00:30:26.269828 exec-cmd.c:243          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core

00:30:26.269828 git.c:748               trace: exec: git-lfs pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git

00:30:26.269828 run-command.c:661       trace: run_command: git-lfs pre-push origin git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git

00:30:26.284632 trace git-lfs: exec: git 'version'

00:30:26.310622 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote' '-v'

00:30:26.332469 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'

00:30:26.348189 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'

00:30:26.364258 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'

00:30:26.381508 trace git-lfs: exec: uname

00:30:26.428441 trace git-lfs: exec: git 'config' '--includes' '-l'

00:30:26.443715 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'

00:30:26.459701 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'

00:30:26.475625 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'

00:30:26.492182 trace git-lfs: attempting pure SSH protocol connection

00:30:26.492708 trace git-lfs: run_command: ssh -oControlMaster=auto -oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-3854338605\sock-%C git@bitbucket.org git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload

00:30:26.496480 trace git-lfs: exec: ssh '-oControlMaster=auto' '-oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-3854338605\sock-%C' 'git@bitbucket.org' 'git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload'

00:30:26.523235 trace git-lfs: pure SSH protocol connection failed: Unable to negotiate version with remote side (unable to read capabilities): EOF

00:30:26.523888 trace git-lfs: pre-push: refs/heads/master 7fa39d52acb51d4e34393b3b6afc559fb343da2d refs/heads/master 5bf600c8130a63a81f909f62d281108acecebc6d

00:30:26.523888 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'show-ref'

00:30:26.554562 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'ls-remote' '--heads' '--tags' '-q' 'origin'

00:30:30.118578 trace git-lfs: attempting pure SSH protocol connection

00:30:30.118578 trace git-lfs: run_command: ssh -oControlMaster=auto -oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-3187652663\sock-%C git@bitbucket.org git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload

00:30:30.122456 trace git-lfs: exec: ssh '-oControlMaster=auto' '-oControlPath=C:\Users\MICHAE~1\AppData\Local\Temp\sock-3187652663\sock-%C' 'git@bitbucket.org' 'git-lfs-transfer michael_thx/lfs-auth-ssh-issue.git upload'

00:30:30.149989 trace git-lfs: pure SSH protocol connection failed: Unable to negotiate version with remote side (unable to read capabilities): EOF

00:30:30.149989 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

00:30:30.153720 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

client_loop: send disconnect: Connection reset

06:17:17.544048 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: client_loop: send disconnect: Connection reset (try: 0/6)

06:17:17.544048 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:17:17.561435 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:18:20.599643 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 1/6)

06:18:20.599918 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:18:20.606454 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:19:22.522672 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 2/6)

06:19:22.522672 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:19:22.527056 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:20:24.471788 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 3/6)

06:20:24.472168 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:20:24.477505 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:21:26.371207 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 4/6)

06:21:26.371394 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:21:26.375917 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:22:28.285033 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 5/6)

Remote "origin" does not support the Git LFS locking API. Consider disabling it with:

  $ git config lfs.https://bitbucket.org/michael_thx/lfs-auth-ssh-issue.git/info/lfs.locksverify false

06:22:28.285286 trace git-lfs: tq: running as batched queue, batch size of 100

06:22:28.285792 trace git-lfs: run_command: git rev-list --objects --ignore-missing --stdin --

06:22:28.286307 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-list' '--objects' '--ignore-missing' '--stdin' '--'

06:22:28.296764 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'cat-file' '--batch-check'

06:22:28.299105 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-common-dir'

06:22:28.328367 trace git-lfs: tq: sending batch of size 1

06:22:28.328434 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:22:28.332131 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:23:30.226909 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 0/6)

06:23:30.227101 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:23:30.230604 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:24:32.160284 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 1/6)

06:24:32.160284 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:24:32.164438 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:25:34.102833 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 2/6)

06:25:34.102833 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:25:34.107056 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:26:36.032464 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 3/6)

06:26:36.032867 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:26:36.037263 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:27:37.973663 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 4/6)

06:27:37.973888 trace git-lfs: run_command: ssh git@bitbucket.org git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload

06:27:37.977955 trace git-lfs: exec: ssh 'git@bitbucket.org' 'git-lfs-authenticate michael_thx/lfs-auth-ssh-issue.git upload'

06:28:39.882160 trace git-lfs: ssh: git@bitbucket.org failed, error: exit status 255, message: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey). (try: 5/6)

batch request: sign_and_send_pubkey: signing failed for RSA "(none)" from agent: agent refused operation

git@bitbucket.org: Permission denied (publickey).: exit status 255

Uploading LFS objects:   0% (0/1), 0 B | 0 B/s, done.

error: failed to push some refs to 'bitbucket.org:michael_thx/lfs-auth-ssh-issue.git'

fatal: unable to write flush packet: Broken pipe

C:\src\lfs-ssh-issue\lfs-auth-ssh-issue [master ↑2]> kkejknhdipp^C

C:\src\lfs-ssh-issue\lfs-auth-ssh-issue [master ↑2]> git status

12:54:29.932442 exec-cmd.c:243          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin

12:54:29.932442 git.c:462               trace: built-in: git status

On branch master

Your branch is ahead of 'origin/master' by 2 commits.

  (use "git push" to publish your local commits)

nothing to commit, working tree clean

C:\src\lfs-ssh-issue\lfs-auth-ssh-issue [master ↑2]> git remote -v

12:54:35.090542 exec-cmd.c:243          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin

12:54:35.092608 git.c:462               trace: built-in: git remote -v

origin  git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git (fetch)

origin  git@bitbucket.org:michael_thx/lfs-auth-ssh-issue.git (push)

C:\src\lfs-ssh-issue\lfs-auth-ssh-issue [master ↑2]>

[Mark C · Accepted solution]

Hi Everyone,

We are happy to advise that we received an update from our engineering team that Git LFS degradation is now fixed.

We apologize for the inconvenience this has caused you and your team.

Regards,
Bitbucket Cloud Support Team

[Aman Chandra Pandey]

14:07:32.114483 trace git-lfs: HTTP: 408

< HTTP/2.0 408 Request Timeout
< Atl-Traceid: 87768c0b3bf05890
< Date: Thu, 19 Oct 2023 08:37:32 GMT
< Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
< Report-To: {"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
< Server: AtlassianEdge
< Strict-Transport-Security: max-age=63072000; preload
< Vary: Accept-Encoding
< X-Content-Type-Options: nosniff
< X-Xss-Protection: 1; mode=block
<
14:07:32.114944 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "a59d4c4cc0216aa7cb031117668dcd97cdf8e5d3880d0109fefc3ef1618f411d"
14:07:32.115041 trace git-lfs: tq: retrying object a59d4c4cc0216aa7cb031117668dcd97cdf8e5d3880d0109fefc3ef1618f411d: LFS: Client error &{%!!(string=https) %!!(string=) %!!(*url.Userinfo=<nil>) %!!(string=api.media.atlassian.com) %!!(string=/upload/5abcff0d-ddd2-4862-b25c-fa2baabd9d74/binary) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=hashAlgorithm=sha256) %!!(string=) %!!(string=)}s(MISSING) from HTTP 408
14:07:32.115146 trace git-lfs: tq: enqueue retry #2 after 0.50s for "a59d4c4cc0216aa7cb031117668dcd97cdf8e5d3880d0109fefc3ef1618f411d" (size: 1569658995)
14:07:32.615316 trace git-lfs: tq: sending batch of size 1
14:07:32.615488 trace git-lfs: ssh cache expired: git@bitbucket.org git-lfs-authenticate quantrium-tech/bank_statement.git upload
14:07:32.615535 trace git-lfs: run_command: sh -c ssh -vvv git@bitbucket.org 'git-lfs-authenticate <repo_name>.git upload'
14:07:32.615624 trace git-lfs: exec: sh '-c' 'ssh -vvv git@bitbucket.org 'git-lfs-authenticate <repo-name>.git upload''

I am getting issue while uploading a 1.5 gb file with Git LFS. Can you help me regarding this?

[Kiwi]

Workaround by override lfs.url

For anyone else that has this issue I did find a workaround be setting git config lfs.url so that the git client bypasses authentication over ssh and uses http basic auth. Still sucks because it means you have to have either a user app password or repo access key setup which isn't as secure.

The workaround is described at the end of this git lfs documentation page under "Custom Configuration"

git-lfs/docs/api/server-discovery.md at main · git-lfs/git-lfs · GitHub

The correct URL to use is obtained from executing the LFS authentication over ssh from the command line.

ssh git@bitbucket.org git-lfs-authenticate <workspace>/<repo>.git upload

The HREF in the response is the URL to use, and is specific to the repo, so be aware that using the .lfsconfig method and forking the repo without changing the lfs.url will still point back to the origin of the fork.... and LFS objects can only be deleted one at a time in Bitbucket (first hand experience ;)

[Kiwi]

Official Response From Atlassian Bitbucket Support

--- edited ----

Was reposting a reply but I see Mark has updated this post too.

[Jean-Paul]

I'm experiencing somthing similar (on two different computers), I've been debugging the last couple of days to no avail.

I'm using an ssh key with passphrase, a key without a passphrase seems to work fine.

git-lfs/3.3.0 (GitHub; windows amd64; go 1.19.3; git 77deabdf)
git version 2.41.0.windows.2
OpenSSH_for_Windows_9.2p1, LibreSSL 3.7.2

My setup was working fine recentely but now I'm also seeing the following command hang.

D:\test>git clone git@bitbucket.org:steelsimvr/steelsimvr.devops.git
...
...
...
trace git-lfs: pure SSH protocol connection failed: Unable to negotiate version with remote side (unable to read capabilities): EOF
...
...
trace git-lfs: run_command: sh -c ssh -vvv git@bitbucket.org 'git-lfs-authenticate <...>/<...>.git download'
trace git-lfs: exec: sh '-c' 'ssh -vvv git@bitbucket.org 'git-lfs-authenticate <...>/<...>.git download''

What I've noticed, and find suspicious, is that when I run the following command directly, the commandline only continues after pressing enter. In other words, it does not return to the prompt only after hitting enter.

@Kiwiis this the case with your setup too?

D:\test>ssh git@bitbucket.org git-lfs-authenticate <...>/<...>.git download

[Kiwi]

@Jean-Paul I see the exact same behavior with bitbucket.org. In contrast, executing

ssh git@github.com git-lfs-authenticate <some repo> upload

returns without any interaction.

[Kiwi]

@Jean-Paul I've created a bitbucket support ticket that links back to our comments in this thread. I suggest you do the same.

[William Pfeffer]

Started to run into the same issue today. Changing the `GIT_SSH` path to point to the Git installed `ssh.exe` instead of the OpenSSH `ssh.exe` resolved it.

[Kiwi]

Interesting. I'm running GitForWindows (Git for Windows (github.com) and git-for-windows/git-for-windows.github.io: Git for Windows' home page) and I don't have an ssh distributed with git. I'm using openssh for Windows (versions in my original post).

What exact path are you using?

[Jean-Paul]

Indeed this works (this is now my workaround), however you cannot use a password protected key. The benefit of using the windows openssh version is that you can add the password to the ssh-agent service.

(The built in ssh.exe mentioned above is an option while installing git for windows)

[William Pfeffer]

The path I'm using is `C:\Program Files\Git\usr\bin\ssh.exe`. I'm using git version 2.41.0.windows.2

[Kiwi]

Interesting on 2 counts.... I'm using the same version of git but I don't get a mingw version of ssh installed.

C:\Users\Kiwi> git --version
git version 2.41.0.windows.2
C:\Users\Kiwi> ls 'C:\Program Files\Git\usr\bin\ssh*.exe'

Directory: C:\Program Files\Git\usr\bin

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 7/7/2023 09:21 22634 ssh-pageant.exe

C:\Users\Kiwi>

[Jean-Paul]

Try reinstalling git for windows, make sure to check the use built in option. (First option)

[Kiwi]

Thanks @Jean-Paul. I did get the mingw ssh installed. Sadly, it's still not a viable workaround for me. My ssh keys are not stored in files, and even if they were, open (no password) private keys isn't something I can do.