Newbie to Bitbucket Pipelines: Zip a repo and send to sftp site?

[Russ Urquhart]

Like I said in the header, I'm a newbie to BitBucket pipelines. I need to write a pipeline that will zip up my current repo, then take that zip file and send it via sftp to our doc host. Does anyone have a sample script that might do some of those things? Or some docs talking about doing something like that?

Thaks,

Russ

[Ben]

G'day Russ!

Welcome to the Bitbucket Cloud community :)

As the pipeline is executed on a branch, the clone directory will contain all of the src files that belong to that branch and this will be the current working directory when you execute a build.

If you were to compress that branch, you could add the following command in your YML file to compress all of the files within the clone directory to a zip folder using zip command with recursive option:

- zip -r branchname.zip .

I would then suggest using the sftp-deploy pipe as documented below to send the resulting zip folder to your sftp server:
https://bitbucket.org/atlassian/sftp-deploy/src/master/

More information regarding pipes can be found here:
https://support.atlassian.com/bitbucket-cloud/docs/what-are-pipes/

Hope this helps.

Cheers!

- Ben (Bitbucket Cloud Support)

[Russ Urquhart]

Thank you I will look at this. Is the pipes.yaml file, is that located in the same branch as well?

[Ben]

Hey Russ,

Pipes are configured in the same bitbucket-pipelines.yml file as the rest of the build steps. This file is located in the same branch that you are executing the build steps :)

Cheers!

- Ben (Bitbucket Cloud Support)

[Russ Urquhart]

I finally got back around to this. I wrote this pipeline to zip my current repo and then send to our host site. We have an SSH key associated with this host provider. Does this pipeline look like it will work:

# This is an example Starter pipeline configuration
# Use a skeleton to build, test and deploy using manual and parallel steps
# -----


pipelines:
  default:
    - parallel:
      - step:
          name: 'Test sending zip file to zoomin'
          script:
            - zip -r bundle.zip .
          - pipe: atlassian/sftp-deploy:0.5.11
          variables:
            USER: 'boomi'
            SERVER: 'upload.zoominsoftware.io'
            REMOTE_PATH: 'boomi-be-staging.zoominsoftware.io/dita-vasont/incoming'
            LOCAL_PATH: 'bundle.zip'
            SSH_KEY: $BITBUCKET_SSH_KEY_FILE

[Ben]

Hey Russ,

Provided the SSH key is correct this should be fine. I would suggest removing the parallel tag however as it is not needed, the zip command will be executed prior to running the pipe.

Please test this and let me know how it goes.

Cheers!

- Ben (Bitbucket Cloud Support)

[Russ Urquhart]

I had to add some stuff, to add zip functionality. 

pipelines:
  default:
    - step:
        name: 'Test sending zip file to zoomin'
        script:
          - sudo apt-get update
          - sudo apt-get install zip
          - zip -r bundle.zip .
          - pipe: atlassian/sftp-deploy:0.5.11
            variables:
            USER: '*****'
            SERVER: '*****'
            REMOTE_PATH: '*****'
            LOCAL_PATH: 'bundle.zip'
            SSH_KEY: $BITBUCKET_SSH_KEY_FILE


I am currently getting Hosting Authentication error. I made sure I put my private SSH key in the BITBUCKET_SSH_KEY_FILE repository variable. Got the same error. I made sure I did a base64 encode of my private key, then put THAT into the same variable. Same thing. When I pasted the private key with the linebreaks, I got a different error, so I know it is accessing the BITBUCKET_SSH_KEY_FILE.

I was able to sftp to that server and put a zip file in the desired location with no problem.

Is it possible that, the manner/encoding that BitBucket is expecting the private SSH key is not compatible with what server is expecting, and so it fails then?

Here is a screenshot:

Also should bundle.zip be saved an an artifact to be passed to the sftp pipe?

Any help is REALLY appreciated! 

Thank you so much!

Russ

[Russ Urquhart]

This is still not working, can someone help me?

[Ben]

Hey Russ,

I have once again censored your output due to this being a public forum.

You will need to ensure that you generate an RSA SSH key with no passphrase using the following command:

ssh-keygen -t rsa -b 4096 -N '' -f my_ssh_key

Once complete, you will then need to encode the key in Base64:

base64 < my_ssh_key

Once done, you will then need to copy paste this into your repository variables.

After completion, you need to install the public SSH key on your remote server and add a my_known_hosts file to your Bitbucket Repo and commit it:

ssh-keyscan -t rsa server.example.com > my_known_hosts

A complete guide can be found here under "use multiple SSH keys in your pipeline":

https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/#Use-multiple-SSH-keys-in-your-pipeline

Please follow the guide above step-by-step. Once complete, please let me know if you run into any further issues.

Cheers!

- Ben (Bitbucket Cloud Support)

[Russ Urquhart]

Hi Ben,

my key was generated with 

`ssh-keygen -t rsa -b 4096 -m PEM`

giving a `id_rsa` and `id_rsa.pub`.  This key has no passphrase. Is this format ok?

I have done the base64 encoding, but will try it again. Do both the private AND public key need to be base64 encoded prior to putting their values into the Repository settings>SSH key section?

My public key is already on the remote server, as I can sftp from the command line.

My `~/.ssh` directory has a `known_hosts` file. I will add a unique `my_known_hosts` file on the repo.

I will give this a try and let you know!

Thanks again!

Russ