Hi, i was trying to get a pretty Simple Pipeline going on a Self Hosted Runner. The idea was to simply copy the content of the repo into a dedicated Folder on a Server in the same network as the Runner but it is not accessible via the Internet. The issue i am having is that it is not documented at all on how to add a internal host into the known_hosts file or even "who" initiates the connection. If it's the Runner, it is able to reach the Target Host on the network standpoint but the logs are only displaying "no route to host" which confuses me. pipelines: custom: customPipelineWithRunnerStep: - step: deployment: Test name: Step 1 services: - docker runs-on: - 'self.hosted' script: - echo "$USER;$SERVER"; - pipe: atlassian/scp-deploy:1.2.1 variables: USER: $USER SERVER: $SERVER REMOTE_PATH: '/var/www/' LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/*' DEBUG: 'true' caches: - docker pipe: atlassian/scp-deploy:1.2.1 + docker container run \ --volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \ --volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \ --volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \ --volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \ --volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy \ --workdir=$(pwd) \ --label=org.bitbucket.pipelines.system=true \ --env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \ --env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \ --env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \ --env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \ --env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \ --env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \ --env=BITBUCKET_DEPLOYMENT_ENVIRONMENT_UUID="$BITBUCKET_DEPLOYMENT_ENVIRONMENT_UUID" \ --env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \ --env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \ --env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \ --env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \ --env=BITBUCKET_DEPLOYMENT_ENVIRONMENT="$BITBUCKET_DEPLOYMENT_ENVIRONMENT" \ --env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \ --env=CI="$CI" \ --env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \ --env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \ --env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \ --env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \ --env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \ --env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \ --env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \ --env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \ --env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \ --env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \ --env=DOCKER_HOST="tcp://host.docker.internal:2375" \ --env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \ --env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy" \ --env=LOCAL_PATH="${BITBUCKET_CLONE_DIR}/*" \ --env=REMOTE_PATH="/var/www/" \ --env=SERVER="$SERVER" \ --env=USER="$USER" \ --add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \ bitbucketpipelines/scp-deploy:1.2.1 Unable to find image 'bitbucketpipelines/scp-deploy:1.2.1' locally 1.2.1: Pulling from bitbucketpipelines/scp-deploy bb7d5a84853b: Pulling fs layer f02b617c6a8c: Pulling fs layer d32e17419b7e: Pulling fs layer c9d2d81226a4: Pulling fs layer 3c24ae8b6604: Pulling fs layer 8a4322d1621d: Pulling fs layer a03ef301ddd7: Pulling fs layer a4c591fc96f3: Pulling fs layer c2fde97fe1fb: Pulling fs layer 02d081902850: Pulling fs layer 4e6a5cdfcfa7: Pulling fs layer bad96da9f98d: Pulling fs layer 1a9bd9b01ebe: Pulling fs layer 8d8633d00c54: Pulling fs layer c9d2d81226a4: Waiting 3c24ae8b6604: Waiting 8a4322d1621d: Waiting a03ef301ddd7: Waiting a4c591fc96f3: Waiting c2fde97fe1fb: Waiting 02d081902850: Waiting 4e6a5cdfcfa7: Waiting bad96da9f98d: Waiting 1a9bd9b01ebe: Waiting 8d8633d00c54: Waiting f02b617c6a8c: Verifying Checksum f02b617c6a8c: Download complete d32e17419b7e: Verifying Checksum d32e17419b7e: Download complete bb7d5a84853b: Verifying Checksum bb7d5a84853b: Download complete bb7d5a84853b: Pull complete f02b617c6a8c: Pull complete d32e17419b7e: Pull complete 8a4322d1621d: Verifying Checksum 8a4322d1621d: Download complete c9d2d81226a4: Verifying Checksum c9d2d81226a4: Download complete 3c24ae8b6604: Verifying Checksum 3c24ae8b6604: Download complete a4c591fc96f3: Verifying Checksum a4c591fc96f3: Download complete c9d2d81226a4: Pull complete a03ef301ddd7: Verifying Checksum a03ef301ddd7: Download complete 02d081902850: Verifying Checksum 02d081902850: Download complete 4e6a5cdfcfa7: Verifying Checksum 4e6a5cdfcfa7: Download complete c2fde97fe1fb: Verifying Checksum c2fde97fe1fb: Download complete 3c24ae8b6604: Pull complete 8a4322d1621d: Pull complete a03ef301ddd7: Pull complete a4c591fc96f3: Pull complete c2fde97fe1fb: Pull complete 02d081902850: Pull complete 4e6a5cdfcfa7: Pull complete bad96da9f98d: Verifying Checksum bad96da9f98d: Download complete 1a9bd9b01ebe: Verifying Checksum 1a9bd9b01ebe: Download complete bad96da9f98d: Pull complete 8d8633d00c54: Verifying Checksum 8d8633d00c54: Download complete 1a9bd9b01ebe: Pull complete 8d8633d00c54: Pull complete Digest: sha256:b9111f61b5824ca7ed1cb63689a6da55ca6d6e8985eb778c36a5dfc2ffe776a8 Status: Downloaded newer image for bitbucketpipelines/scp-deploy:1.2.1 INFO: Configuring ssh with default ssh key. INFO: Adding known hosts... INFO: Appending to ssh config file private key path INFO: Applied file permissions to ssh directory. ssh: connect to host 172.18.1.28 port 22: No route to host lost connection ✖ Deployment failed. Could someone support me here?

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

scp-deploy from a Self Hosted runner to Internal Server

Bao Le_53o022

Hi,

i was trying to get a pretty Simple Pipeline going on a Self Hosted Runner. The idea was to simply copy the content of the repo into a dedicated Folder on a Server in the same network as the Runner but it is not accessible via the Internet.

The issue i am having is that it is not documented at all on how to add a internal host into the known_hosts file or even "who" initiates the connection. If it's the Runner, it is able to reach the Target Host on the network standpoint but the logs are only displaying "no route to host" which confuses me.

pipelines:

  custom:

    customPipelineWithRunnerStep:

      - step:

          deployment: Test

          name: Step 1

          services:

            - docker

          runs-on: 

            - 'self.hosted'

          script:

            - echo "$USER;$SERVER";

            - pipe: atlassian/scp-deploy:1.2.1

              variables:

                USER: $USER

                SERVER: $SERVER

                REMOTE_PATH: '/var/www/'

                LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/*'

                DEBUG: 'true'

          caches:

            - docker

pipe: atlassian/scp-deploy:1.2.1 + docker container run \
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \
--volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy \
--workdir=$(pwd) \
--label=org.bitbucket.pipelines.system=true \
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \
--env=BITBUCKET_DEPLOYMENT_ENVIRONMENT_UUID="$BITBUCKET_DEPLOYMENT_ENVIRONMENT_UUID" \
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \
--env=BITBUCKET_DEPLOYMENT_ENVIRONMENT="$BITBUCKET_DEPLOYMENT_ENVIRONMENT" \
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \
--env=CI="$CI" \
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \
--env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \
--env=DOCKER_HOST="tcp://host.docker.internal:2375" \
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/scp-deploy" \
--env=LOCAL_PATH="${BITBUCKET_CLONE_DIR}/*" \
--env=REMOTE_PATH="/var/www/" \
--env=SERVER="$SERVER" \
--env=USER="$USER" \
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \
bitbucketpipelines/scp-deploy:1.2.1
Unable to find image 'bitbucketpipelines/scp-deploy:1.2.1' locally
1.2.1: Pulling from bitbucketpipelines/scp-deploy
bb7d5a84853b: Pulling fs layer
f02b617c6a8c: Pulling fs layer
d32e17419b7e: Pulling fs layer
c9d2d81226a4: Pulling fs layer
3c24ae8b6604: Pulling fs layer
8a4322d1621d: Pulling fs layer
a03ef301ddd7: Pulling fs layer
a4c591fc96f3: Pulling fs layer
c2fde97fe1fb: Pulling fs layer
02d081902850: Pulling fs layer
4e6a5cdfcfa7: Pulling fs layer
bad96da9f98d: Pulling fs layer
1a9bd9b01ebe: Pulling fs layer
8d8633d00c54: Pulling fs layer
c9d2d81226a4: Waiting
3c24ae8b6604: Waiting
8a4322d1621d: Waiting
a03ef301ddd7: Waiting
a4c591fc96f3: Waiting
c2fde97fe1fb: Waiting
02d081902850: Waiting
4e6a5cdfcfa7: Waiting
bad96da9f98d: Waiting
1a9bd9b01ebe: Waiting
8d8633d00c54: Waiting
f02b617c6a8c: Verifying Checksum
f02b617c6a8c: Download complete
d32e17419b7e: Verifying Checksum
d32e17419b7e: Download complete
bb7d5a84853b: Verifying Checksum
bb7d5a84853b: Download complete
bb7d5a84853b: Pull complete
f02b617c6a8c: Pull complete
d32e17419b7e: Pull complete
8a4322d1621d: Verifying Checksum
8a4322d1621d: Download complete
c9d2d81226a4: Verifying Checksum
c9d2d81226a4: Download complete
3c24ae8b6604: Verifying Checksum
3c24ae8b6604: Download complete
a4c591fc96f3: Verifying Checksum
a4c591fc96f3: Download complete
c9d2d81226a4: Pull complete
a03ef301ddd7: Verifying Checksum
a03ef301ddd7: Download complete
02d081902850: Verifying Checksum
02d081902850: Download complete
4e6a5cdfcfa7: Verifying Checksum
4e6a5cdfcfa7: Download complete
c2fde97fe1fb: Verifying Checksum
c2fde97fe1fb: Download complete
3c24ae8b6604: Pull complete
8a4322d1621d: Pull complete
a03ef301ddd7: Pull complete
a4c591fc96f3: Pull complete
c2fde97fe1fb: Pull complete
02d081902850: Pull complete
4e6a5cdfcfa7: Pull complete
bad96da9f98d: Verifying Checksum
bad96da9f98d: Download complete
1a9bd9b01ebe: Verifying Checksum
1a9bd9b01ebe: Download complete
bad96da9f98d: Pull complete
8d8633d00c54: Verifying Checksum
8d8633d00c54: Download complete
1a9bd9b01ebe: Pull complete
8d8633d00c54: Pull complete
Digest: sha256:b9111f61b5824ca7ed1cb63689a6da55ca6d6e8985eb778c36a5dfc2ffe776a8
Status: Downloaded newer image for bitbucketpipelines/scp-deploy:1.2.1
INFO: Configuring ssh with default ssh key.
INFO: Adding known hosts...
INFO: Appending to ssh config file private key path
INFO: Applied file permissions to ssh directory.
ssh: connect to host 172.18.1.28 port 22: No route to host
lost connection
✖ Deployment failed.

Could someone support me here?

Accepted answers

Patrik S

Hello @Bao Le ,

Welcome to Atlassian Community!

The error message being shown "connect to host 172.18.1.28 port 22: No route to host" usually happens when the destination IP has a firewall blocking the SSH traffic, such as iptables.

In order to check if the destination IP is listening to port 22, could you please execute the command below in the host you are running the runner and share the log with us?

nc -vz 172.18.1.28 22

Also, if you try to use the same scp command outside of the runner, executing it directly in the host machine and to the same destination IP, is the command successful ?

Let me know if you have any question.

Thank you, @Bao Le .

Kind regards,

Patrik S

All comments

Patrik S

Hello @Bao Le ,

Welcome to Atlassian Community!

The error message being shown "connect to host 172.18.1.28 port 22: No route to host" usually happens when the destination IP has a firewall blocking the SSH traffic, such as iptables.

In order to check if the destination IP is listening to port 22, could you please execute the command below in the host you are running the runner and share the log with us?

nc -vz 172.18.1.28 22

Also, if you try to use the same scp command outside of the runner, executing it directly in the host machine and to the same destination IP, is the command successful ?

Let me know if you have any question.

Thank you, @Bao Le .

Kind regards,

Patrik S

Bao Le_53o022

Hi @Patrik S ,

thank you for your response.
Basically i have already tried to directly ssh to the target host which succeeds.

Running your command results in the following


nc -vz 172.18.1.28 22
172.18.1.28: inverse host lookup failed: Unknown host
(UNKNOWN) [172.18.1.28] 22 (ssh) open

When directly doing ssh 172.18.1.28 from the runner i get this

ssh 172.18.1.28
The authenticity of host '172.18.1.28 (172.18.1.28)' can't be established.
ED25519 key fingerprint is SHA256:is3d7QUtkEBf3ATPgGmhRy4a1zsTUNZzGYEaOZE3Nes.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

this is expected because i haven't actually added the target host to the known hosts list so far.

Going scp is also working fine.

How can i proceed here?

Patrik S

Hello @Bao Le ,

Thank you for getting back with the results.

The nc command shows that indeed the destination server is listening to port 22.

I would like to ask you trying to connect over ssh with verbose logging and doing scp without using the pipe, by including the below commands to your bitbucket-pipelines.yml file before the pipe:

ssh -Tvvv $SERVER -o StrictHostKeyChecking=no

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -r -v $BITBUCKET_CLONE_DIR $USER@$SERVER:/var/www/

The above commands will temporarily disable the host key checking, so you are not prompted to confirm the thumbprint.

Once you add the above commands and run your pipeline again, please share us the log output, making sure to sanitize any sensitive information.

Thank you, @Bao Le .

Kind regards,

Patrik S

Bao Le_53o022

Hi @Patrik S ,

that is my forth attempt in another Browser because the editor seems to find some links in my text... even tho there is none.

Anyways, here is the Result.

ssh -Tvvv $SERVER -o StrictHostKeyChecking=no
+ ssh -Tvvv $SERVER -o StrictHostKeyChecking=no
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 172.18.1.28 [172.18.1.28] port 22.
debug1: connect to address 172.18.1.28 port 22: No route to host
ssh: connect to host 172.18.1.28 port 22: No route to host

And here the Pipelines file

PS: This editor is the worst i have seen since a looong time.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

server

bitbucket

scp

bitbucket-runners