SCP-Deploy Broken Pipe issue less than a minute after pipe starts

Question

I'm using SCP-Deploy with OpenSSH on Windows Server. Everything worked fine for the first few runs. Suddenly I started getting Broken Pipe issue and scp proces freezes. The pipeline doesn't stop, looks like it's still going, no error is shown, but nothing happens and I have to stop it manually as it never progresses:

scp: E:/Sites/MYAPI/Microsoft.AspNetCore.Identity.EntityFrameworkCore.dll: Broken pipe

This happened intermittently, now happens pretty much every time I run the pipe. Tried setting ServerAliveInterval etc, but this doesn't help the issue. Any idea why this could be the case? SCP is my only option, apart from slow SFTP to move files to Windows Server from Bitbucket Pipelines hence any help would be greatly appreciated. Files I'm moving are not huge, it's a 200mb in total of .NET API where no one file is extremely large. The Broken Pipe issue shows up pretty much at the beginning of the transfer.

SCP yml section:

- pipe: atlassian/scp-deploy:1.2.1
  variables:
    USER: $WEB2_USER
    SERVER: $WEB2_SERVER
    SSH_PORT: '22'
    REMOTE_PATH: $WEB2_PATH/Services
    LOCAL_PATH: /opt/atlassian/pipelines/agent/build/deploy/Services/*
    EXTRA_ARGS: ['-o', 'ServerAliveInterval=60', '-o', 'ServerAliveCountMax=10']
    DEBUG: 'true'

There is no difference when I remove EXTRA_ARGS

kamilkloc · Answer

Yes, solution was to move to Azure Pipelines. Bitbucket has very very poor support for Windows deployment and it is just more painful and hacky to set up for it to work than its worth it. Intermittent issues keep happening,  bitbucket Staff keeps saying the problem is on my side... well, we tried 3 different Windows machines, same results. In Azure we set up our Windows machine as User Agent and it works smoothly and downloads the repo on the server level, rather than having to SCP it onto server. Bit bucket pipelines we're fine to learn how pipelines work, that's it, sadly.

Kent Hosbjerg_y65fxg · Answer

Hi @kamilkloc ,

We are experiencing the same issue.

We upgraded scp-deploy version 0.3.0 > 1.2.1 because we thought it was a version issue. But, the error is still there.

Did you find a solution to the problem ?

kamilkloc · Answer

Might be on my side, or OpenSSH side, I checked firewall etc, all seems fine, no idea what it could be to be honest, especially that it worked fine multiple times beforehand.

One thing that is bugging me is the fact that it only says "broken pipe" and bitbucket pipeline doesn't stop - seems like it's still running indefinitely (although not progressing) and has to be manually interrupted. Debug option doesn't show anything neither, no other messages apart from broken pipe next to the random file when it tries pushing it to the server.