"Connection reset by peer in connection to bitbucket.org:443" in Pipelines

Rudolf Byker

I'm using a self-hosted runner, version 1.369 (the latest at the time of writing). It was working well for a long time, but recently, without any infrastructure changes, I started getting this error on all steps:

We couldn't clone the repository. Try rerunning the pipeline.

In the "build setup", I see this output:

+ umask 000
+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="..." https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 
Attempt 1 of 6 failed.
Retrying in 1 seconds
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 
Attempt 2 of 6 failed.
Retrying in 2 seconds

etc...

The setup

This runner is in kubernetes on a K3s cluster. It's running on a development node which is a physical machine in my house. I manage this cluster, and nothing changed in the network.

The deployment for my runner instance is as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
namespace: bbp-runner
name: runner
labels:
app: bbp-runner
spec:
replicas: 1
selector:
matchLabels:
app: bbp-runner
template:
metadata:
labels:
app: bbp-runner
accountUuid: foo
runnerUuid: bar
spec:
nodeSelector:
env: prod
containers:
        - name: bitbucket-k8s-runner
image: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
imagePullPolicy: Always
env:
            - name: ACCOUNT_UUID
value: "{foo}"
            - name: RUNNER_UUID
value: "{bar}"
            - name: OAUTH_CLIENT_ID
valueFrom:
secretKeyRef:
name: runner-oauth-credentials
key: oauthClientId
            - name: OAUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: runner-oauth-credentials
key: oauthClientSecret
            - name: WORKING_DIRECTORY
value: "/tmp"
          volumeMounts:
            - name: tmp
mountPath: /tmp
            - name: docker-containers
mountPath: /var/lib/docker/containers
readOnly: true
            - name: var-run
mountPath: /var/run
        - name: docker-in-docker
image: docker:20.10.7-dind
securityContext:
privileged: true
volumeMounts:
            - name: tmp
mountPath: /tmp
            - name: docker-containers
mountPath: /var/lib/docker/containers
            - name: var-run
mountPath: /var/run
volumes:
        - name: tmp
emptyDir: {}
        - name: docker-containers
emptyDir: {}
        - name: var-run
emptyDir: {}

The relevant containers are

docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner

and

docker:20.10.7-dind

I followed this guide https://janosmiko.com/blog/2021-09-08-bitbucket-pipelines-runners-in-k8s/ when I set it up a few months ago.

Syahrul

Hey @Rudolf Byker 

G'day.

The error below means the TLS handshake failed to establish the connection.

OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 "

This can happen for multiple reasons, but generally, it's caused by the local network.

I suggest checking with your network team to see if there's any recent change in your Firewall/proxy.

That said, you can also investigate this by running a few tests, such as:

1. Try cloning a repository from your Runner server with verbose logs enable to get more information

   GIT_TRACE_PACKET=1 GIT_TRACE=1 GIT_CURL_VERBOSE=1 <git command_here>
   

2. Run Curl to see if there's any issue with TLS connection from your server 

   curl -v https://bitbucket.org

If both tests failed with the same runner error, then most likely, the connection is being blocked on your Firewall/Proxy level.

I hope this helps.

Cheers,
Syahrul