nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

setup a piple line with sftp copy file

unknown

I have a pipe line with 2 steps , i have create ssh and added public key to my server

- step:
name: deployment
runs-on:
     - 'self.hosted'
    script:
    - pipe: atlassian/ssh-run:0.4.0
variables:
SSH_USER: 'ubuntu'
       SERVER: 172.0.1.195
MODE: 'command'
       DEBUG: 'true'
       COMMAND: 'echo ${BITBUCKET_COMMIT::7} > a.txt'
    - pipe: atlassian/sftp-deploy:0.5.11
variables:
USER: 'ubuntu'
        SERVER: '172.0.1.195'
        REMOTE_PATH: '/home/ubuntu'
        DEBUG: 'true'

but when i tried to execute this pipeline ssh pass and sftp or scp always fail with the following logs

Runner matching labels:
- linux
- self.hosted
Runner name: ta3meed-sandbox-runner
Runner labels: self.hosted, linux
Runner version:
current: 1.333
latest: 1.336

[93mThe version of this runner is outdated. Upgrade to the latest version (1.336). [0m
+ umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="deployment-pipeline" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard bc6100786f45d22fc8ef283281d49687c8e08409
HEAD is now at bc61007 add debug

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

+ git remote set-url origin http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME

+ git reflog expire --expire=all --all

+ echo ".bitbucket/pipelines/generated" >> .git/info/exclude

+ chmod 777 $BUILD_DIR

Images used:
build: atlassian/default-image@sha256:3a09dfec7e36fe99e3910714c5646be6302ccbca204d38539a07f0c2cb5902d4
docker: docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-docker-daemon@sha256:03bfa3875b810fe280239175f742a20c500d9b1f181c83a3b05cdc04b1cfb12c

+ docker container run \
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \
--volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy \
--workdir=$(pwd) \
--label=org.bitbucket.pipelines.system=true \
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \
--env=CI="$CI" \
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \
--env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \
--env=DOCKER_HOST="tcp://host.docker.internal:2375" \
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/sftp-deploy" \
--env=DEBUG="true" \
--env=REMOTE_PATH="/home/ubuntu" \
--env=SERVER="172.0.1.195" \
--env=USER="ubuntu" \
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \
bitbucketpipelines/sftp-deploy:0.5.11
Unable to find image 'bitbucketpipelines/sftp-deploy:0.5.11' locally
0.5.11: Pulling from bitbucketpipelines/sftp-deploy
31603596830f: Already exists
9e006a152763: Pulling fs layer
8c449c3c01f1: Pulling fs layer
1bd77f40acd7: Pulling fs layer
8c449c3c01f1: Verifying Checksum
8c449c3c01f1: Download complete
1bd77f40acd7: Verifying Checksum
1bd77f40acd7: Download complete
9e006a152763: Verifying Checksum
9e006a152763: Download complete
9e006a152763: Pull complete
8c449c3c01f1: Pull complete
1bd77f40acd7: Pull complete
Digest: sha256:9fcd9a8ec73daf0ec51461079b1853fb6b5f93970842f6d80e64160f4b5535ff
Status: Downloaded newer image for bitbucketpipelines/sftp-deploy:0.5.11
+ SFTP_DEBUG_ARGS=-v
+ [[ -z '' ]]
+ info 'Using SSH.'
[36mINFO: Enabling debug mode.[0m
[36mINFO: Using SSH.[0m
+ echo -e '\e[36mINFO: Using SSH.\e[0m'
+ setup_ssh_dir
+ INJECTED_SSH_CONFIG_DIR=/opt/atlassian/pipelines/agent/ssh
+ IDENTITY_FILE=/opt/atlassian/pipelines/agent/ssh/id_rsa_tmp
+ KNOWN_HOSTS_FILE=/opt/atlassian/pipelines/agent/ssh/known_hosts
+ mkdir -p /root/.ssh
+ touch /root/.ssh/authorized_keys
+ [[ -z '' ]]
+ '[' '' '!=' '' ']'
+ '[' '!' -f /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp ']'
+ debug 'Using default ssh key'
+ [[ true == \t\r\u\e ]]
+ echo -e '\e[37mDEBUG: Using default ssh key\e[0m'
+ cp /opt/atlassian/pipelines/agent/ssh/id_rsa_tmp /root/.ssh/pipelines_id
[37mDEBUG: Using default ssh key[0m
+ '[' '!' -f /opt/atlassian/pipelines/agent/ssh/known_hosts ']'
+ cat /opt/atlassian/pipelines/agent/ssh/known_hosts
+ '[' -f /root/.ssh/config ']'
+ [[ -z '' ]]
+ echo 'IdentityFile ~/.ssh/pipelines_id'
+ chmod -R go-rwx /root/.ssh/
+ run_pipe
[36mINFO: Starting SFTP deployment to 172.0.1.195:/home/ubuntu...[0m
[37mDEBUG: Executing echo "mput /opt/atlassian/pipelines/agent/build/*" | sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu[0m
+ info 'Starting SFTP deployment to 172.0.1.195:/home/ubuntu...'
+ echo -e '\e[36mINFO: Starting SFTP deployment to 172.0.1.195:/home/ubuntu...\e[0m'
+ set +e
+ [[ -z '' ]]
+ debug Executing echo '"mput' '/opt/atlassian/pipelines/agent/build/*"' '|' sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu
+ [[ true == \t\r\u\e ]]
+ echo -e '\e[37mDEBUG: Executing echo "mput /opt/atlassian/pipelines/agent/build/*" | sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu\e[0m'
+ echo 'mput /opt/atlassian/pipelines/agent/build/*'
+ sftp -b - -rp -v ubuntu@172.0.1.195:/home/ubuntu
OpenSSH_7.9p1, OpenSSL 1.1.1g 21 Apr 2020
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 172.0.1.195 [172.0.1.195] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/pipelines_id type -1
debug1: identity file /root/.ssh/pipelines_id-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3
debug1: match: OpenSSH_8.9p1 Ubuntu-3 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 172.0.1.195:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:cBkkJ0ADEjHdkR6jqj+XaNobKRAyefvZ4jp/Ub3mPtA
Host key verification failed.

Connection closed.
Connection closed
[31m✖ Deployment failed.[0m
+ STATUS=255
+ set -e
+ [[ 255 == \0 ]]
+ fail 'Deployment failed.'
+ echo -e '\e[31m✖ Deployment failed.\e[0m'
+ exit 255
Searching for files matching artifact pattern .bitbucket/pipelines/generated/pipeline/pipes/**

Searching for test report files in directories named [test-reports, TestResults, test-results, surefire-reports, failsafe-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

Accepted answers

Theodora Boudale

Hi @[deleted]and welcome to the community!

If you open the repository with this pipeline on Bitbucket website, go to its Repsoitory settings > SSH keys, is the address of your server in the Known hosts section?

The error "Host key verification failed" indicates that it may not be there.

If it is not there, can you add it and select Fetch to fetch the host's fingerprint?
After it is successfully added, please run this pipeline again and let me know how it goes.

If you still see errors, please copy-paste the new log so we can further investigate.

Kind regards,
Theodora

All comments

Theodora Boudale

Hi @[deleted]and welcome to the community!

If you open the repository with this pipeline on Bitbucket website, go to its Repsoitory settings > SSH keys, is the address of your server in the Known hosts section?

The error "Host key verification failed" indicates that it may not be there.

If it is not there, can you add it and select Fetch to fetch the host's fingerprint?
After it is successfully added, please run this pipeline again and let me know how it goes.

If you still see errors, please copy-paste the new log so we can further investigate.

Kind regards,
Theodora

unknown

Thanks Theodora for your update , but I am running this pipeline on self-host agent and for local network we are establishing the communication to our application server that we need to copy files their,

as you see IP address is

172.0.1.195

which is reside on the same network of our agent , so it's not applicable to add it to know hosts list.

Theodora Boudale

Hi @[deleted],

Builds that use Linux runners run in a Docker container on your machine, they do not run directly on your machine.

The SSH keys and known_hosts file that you have on your machine with the runner are not accessible and cannot be used by the Docker container that runs your build.

Can you please confirm how you have set up SSH?
Have you generated a new key pair from Repository settings > SSH key?
Have you added your own key pair on that page?
Or do you have an SSH key on the Docker image you are using?

The ~/.ssh/known_hosts file inside the Docker container (not the one on your machine) needs to be updated with the public key of the server you want to send files to. The easiest way to do this is by adding the address of the server on the Known hosts section in the Repository settings of this repo > SSH keys.

Alternatively, you can also try passing an argument to the pipe and set StrictHostKeyChecking to no, to prevent this error without updating known_hosts. You could do this as follows:

- pipe: atlassian/sftp-deploy:0.5.11
  variables:
    USER: 'ubuntu'
    SERVER: '172.0.1.195'
    REMOTE_PATH: '/home/ubuntu'
    DEBUG: 'true'
    EXTRA_ARGS: '-o StrictHostKeyChecking=no'

Kind regards,
Theodora

Igor Stoyanov

@[deleted] hi. Thanks for your question.

Since you are using self hosted runner, you should try to pass your ssh key directly to the pipe:

script:
  - pipe: atlassian/ssh-run:0.4.1
    variables:
      SSH_USER: 'ec2-user'
      SERVER: '127.0.0.1'
      SSH_KEY: $MY_SSH_KEY
      MODE: 'script'
      COMMAND: 'myscript.sh' # path to a script in your repository

SSH_KEY is a base64 encoded alternate SSH_KEY to use instead of the key configured in the Bitbucket Pipelines admin screens (which is used by default). This should be encoded.

To encode ssh key:

Linux

 $ base64 -w 0 < my_ssh_key

Mac OS X

 $ base64 < my_ssh_key

Best regards, Igor.

unknown

Thanks @Theodora Boudale Extra Args solved my problem

unknown

@Igor Stoyanov

ssh Pipe is working fine with me , but ftp pipe has a problem , and @Theodora Boudale helped me to resolve this issue using extra args to all unkhown host names

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

cloud

bitbucket

bitbucket-cloud

bitbucket-pipelines

ssh

scp

sftp