Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
Community Members
Community Events
Community Groups

Upload only changed files using rsync pipe

I'm trying to manage the deployment pipelines of a very large project, made of several hundreds of files. I would like to upload only the ones that have been changed in the last commit. I made a pipeline using atlassian/rsync-deploy but since it is run in a container, the files are moved in the container and they all have a newer date than those in the destination, and so all of them are transferred by rsync. Is it possible to move only the one that have changed? 

2 answers

The best way is to use flag -c and -u and it will update to u only commited files.

0 votes

Hey @stefano_vita . What you want is possible with a little workaround. The pipe supports the EXTRA_ARGS parameter which you can you to pass arbitrary rsync cli options. So you might need to take the following steps in your pipeline:

  1. Get the list of changed file, for example you can run 
    git diff --name-only HEAD HEAD~1
  2. Write the output to a text file
    git diff --name-only HEAD HEAD~1 > change-files.txt
  3. Use the --include-from rsync option to pass list of files to include
    EXTRA_ARGS: "--include-from='changed-files.txt'" --exclude="*"

    Note that you also need to explicitly exclude all files and include only the changed ones.

If what you're actually focused on is getting the files from this build onto the machine then I'd suggest using --ignore-times instead of --include-from / --exclude. You can't be sure your deploying HEAD on top of HEAD~1, especially if you're using builds from multiple branches.

--ignore-times doesn't worked for me :( could you provide maybe full line with EXTRA_ARGS as example?
I would greatly appreciate :)

Thank you.


Suggest an answer

Log in or Sign up to answer

Atlassian Community Events