I'm using a self-hosted runner, version 1.369 (the latest at the time of writing). It was working well for a long time, but recently, without any infrastructure changes, I started getting this error on all steps:
We couldn't clone the repository. Try rerunning the pipeline.
In the "build setup", I see this output:
+ umask 000
+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="..." https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443
Attempt 1 of 6 failed.
Retrying in 1 seconds
Cloning into '/opt/atlassian/pipelines/agent/build'...
fatal: unable to access 'https://bitbucket.org/.../....git/': OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443
Attempt 2 of 6 failed.
Retrying in 2 seconds
etc...
This runner is in kubernetes on a K3s cluster. It's running on a development node which is a physical machine in my house. I manage this cluster, and nothing changed in the network.
The deployment for my runner instance is as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: bbp-runner
name: runner
labels:
app: bbp-runner
spec:
replicas: 1
selector:
matchLabels:
app: bbp-runner
template:
metadata:
labels:
app: bbp-runner
accountUuid: foo
runnerUuid: bar
spec:
nodeSelector:
env: prod
containers:
- name: bitbucket-k8s-runner
image: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
imagePullPolicy: Always
env:
- name: ACCOUNT_UUID
value: "{foo}"
- name: RUNNER_UUID
value: "{bar}"
- name: OAUTH_CLIENT_ID
valueFrom:
secretKeyRef:
name: runner-oauth-credentials
key: oauthClientId
- name: OAUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: runner-oauth-credentials
key: oauthClientSecret
- name: WORKING_DIRECTORY
value: "/tmp"
volumeMounts:
- name: tmp
mountPath: /tmp
- name: docker-containers
mountPath: /var/lib/docker/containers
readOnly: true
- name: var-run
mountPath: /var/run
- name: docker-in-docker
image: docker:20.10.7-dind
securityContext:
privileged: true
volumeMounts:
- name: tmp
mountPath: /tmp
- name: docker-containers
mountPath: /var/lib/docker/containers
- name: var-run
mountPath: /var/run
volumes:
- name: tmp
emptyDir: {}
- name: docker-containers
emptyDir: {}
- name: var-run
emptyDir: {}
The relevant containers are
docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner
and
docker:20.10.7-dind
I followed this guide https://janosmiko.com/blog/2021-09-08-bitbucket-pipelines-runners-in-k8s/ when I set it up a few months ago.
Hey @Rudolf Byker
G'day.
The error below means the TLS handshake failed to establish the connection.
OpenSSL SSL_connect: Connection reset by peer in connection to bitbucket.org:443 "
This can happen for multiple reasons, but generally, it's caused by the local network.
I suggest checking with your network team to see if there's any recent change in your Firewall/proxy.
That said, you can also investigate this by running a few tests, such as:
GIT_TRACE_PACKET=1 GIT_TRACE=1 GIT_CURL_VERBOSE=1 <git command_here>
curl -v https://bitbucket.org
If both tests failed with the same runner error, then most likely, the connection is being blocked on your Firewall/Proxy level.
I hope this helps.
Cheers,
Syahrul
Thanks for your time.
I added some info about the infrastructure setup to the original post above.
I get a shell in the pod by doing this:
kubectl -n bbp-runner exec -it deploy/runner -- /bin/sh
The `docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner` image does not contain `git` or `curl`, but I could add it with `apk`, and then test what you suggested:
apk add curl git openssh
curl -v https://bitbucket.org
This works fine. I get a 200 response with lots of HTML.
git clone git@bitbucket.org:rscdevelopers/living_word_vue.git
This works fine as well. I get the RSA key fingerprint, and then permission denied as expected, since I don't have the keys set up. This is enough to verify that TLS is working.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hey @Rudolf Byker
Thanks for the update.
We may need to investigate this further, so I went ahead and created a support ticket on your behalf. Please review it here.
One of our Support should get back to you within your timezone shortly.
Cheers,
Syahrul
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.