Understanding Available Pipes

[kstiles31]

For the last year i have been using the ftp-deploy:0.7.1 pipe in bitbucket without issue.

In an effort to better manage my server, i have recently created new SFTP users, so i updated my pipe to sftp-deploy:0.10.0.

In testing the SFTP worked, but there is no delete-flag option, so when i removed files from my source, those files remained on the server (delete-flag was working as intended with ftp-deploy:0.7.1.

Next, i got SSH connection working (with sftp-deploy:0.10.0), then switched to the rsync-deploy:0.13.0 pipe.

Again in testing, even with the available delete-flag set to true, rsync does not remove files that have been removed from the source.

To test, i simply removed a info.php file from the root of my application, the pipe is successful, but the info.php file remains on the server, here is the yml configuration i used:

            - pipe: atlassian/rsync-deploy:0.13.0

              variables:

                USER: $USER

                SERVER: $SERVER

                REMOTE_PATH: $REMOTE_PATH

                LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/*'

                DEBUG: 'true'

                EXTRA_ARGS:

                  - "--exclude=${BITBUCKET_CLONE_DIR}/.htaccess"

                  - "--exclude=*.gitignore"

                  - "--exclude=*.vscode/"

Am i doing something wrong? Or simply misunderstand how delete-after works?

The rsync pipe is much faster than the sftp pipe, but to accomplish what i need (haivng the server codebase match the latest commit) i am fine using either.

[Patrik S · Accepted solution]

Hello @kstiles31 ,

and welcome to the Community!

I see you're using a wildcard to define the LOCAL_PATH variable, and this may be causing the shell to expand the paths to the individual files only (instead of the whole directory), making rsync to not delete the files.

According to rsync documentation on the deletion:

--delete
This tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g. lqdirrq or lqdir/rq) without using a wildcard for the directory's contents (e.g. lqdir/*rq) since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files' parent directory

In this case, may I ask you to try without using the wildcard in the  LOCAL_PATH variable and check if the files are deleted ? The pipe definition would look like the following:

- pipe: atlassian/rsync-deploy:0.13.0
   variables:
     USER: $USER
     SERVER: $SERVER
     REMOTE_PATH: $REMOTE_PATH
     LOCAL_PATH: $BITBUCKET_CLONE_DIR
     DEBUG: 'true'
     EXTRA_ARGS:
       - "--exclude=${BITBUCKET_CLONE_DIR}/.htaccess"
       - "--exclude=*.gitignore"
       - "--exclude=*.vscode/"

[kstiles31]

Appreciate the help.

The wildcard was indeed an issue, however using $BITBUCKET_CLONE_DIR did not work either.

I was able to get the pipe working using '${BITBUCKET_CLONE_DIR}/'

I am also using a trailing / at the end of my $REMOTE_PATH

My current working yml is as follows:

            - pipe: atlassian/rsync-deploy:0.13.0

              variables:

                USER: $USER

                SERVER: $SERVER

                REMOTE_PATH: $REMOTE_PATH

                LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/'

                DELETE_FLAG: 'true'

                DEBUG: 'true'

                EXTRA_ARGS:

                  - "--exclude=.gitignore"

                  - "--exclude=.vscode/"

                  - "--verbose"

(I removed a few excludes for clarity)

[Patrik S]

@kstiles31 , glad to hear that it worked :) And thanks for sharing the YAML with the fixed syntax!