Upload only changed files using rsync pipe

[stefano_vita]

I'm trying to manage the deployment pipelines of a very large project, made of several hundreds of files. I would like to upload only the ones that have been changed in the last commit. I made a pipeline using atlassian/rsync-deploy but since it is run in a container, the files are moved in the container and they all have a newer date than those in the destination, and so all of them are transferred by rsync. Is it possible to move only the one that have changed? 

[Raimundas Ra]

The best way is to use flag -c and -u and it will update to u only commited files.

EXTRA_ARGS: '-c --update ${BITBUCKET_CLONE_DIR}/'

[Alexander Zhukov]

Hey @stefano_vita . What you want is possible with a little workaround. The pipe supports the EXTRA_ARGS parameter which you can you to pass arbitrary rsync cli options. So you might need to take the following steps in your pipeline:

1. Get the list of changed file, for example you can run 

   git diff --name-only HEAD HEAD~1

2. Write the output to a text file

   git diff --name-only HEAD HEAD~1 > change-files.txt

3. Use the --include-from rsync option to pass list of files to include

   EXTRA_ARGS: "--include-from='changed-files.txt'" --exclude="*"

   Note that you also need to explicitly exclude all files and include only the changed ones.

[Nick]

If what you're actually focused on is getting the files from this build onto the machine then I'd suggest using --ignore-times instead of --include-from / --exclude. You can't be sure your deploying HEAD on top of HEAD~1, especially if you're using builds from multiple branches.

[Raimundas Ra]

--ignore-times doesn't worked for me :( could you provide maybe full line with EXTRA_ARGS as example?
I would greatly appreciate :)

Thank you.