Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

aws-s3-deploy updates unchanged files

We use the aws-s3-deploy pipe with the following configuration to upload files from the repository to s3. 

-   pipe: atlassian/aws-s3-deploy:0.2.1
AWS_DEFAULT_REGION: eu-central-1
S3_BUCKET: my-bucket
LOCAL_PATH: resources

Even though no of the files in `resources` has changed, the entire folder is uploaded on each deployment.

The files in the given local path are commited in the repository and should also not be affected by any other steps in the deployment.

1 answer

0 votes

Hi @Nicolas . The pipe users the s3 sync CLI under the hood, so it's a bit tricky to sync only changed files. One possible solution might be to use the combination of --exclude/--include options (see the sync command reference for details). You can pass them via the EXTRA_ARGS pipe variable. Basically, you could get the list of changed files from git and include these files and exclude files that were unchanged. 

thanks for the insights. I was confused by the description in the readme

Recursively copies new and updated files from the source local directory to the destination.

maybe this part should be removed then?

Like # people like this

@Nicolas you could also try to add --size-only to the EXTRA_ARGS pipe variable. This will tell the aws s3 sync to only upload files that have changed the size. See for how this option affects the sync command behavior.

your suggested approach wouldn't work as the size can remain unchanged even though the file content has been changed. Please correct me if I am wrong.
I do have the same issue and want to upload/sync only the changed files to S3.

i have the same problem as this, did anyone find a solution to it?  also need the same for the google bucket upload as well



would, this work?

--exclude --exact-timestamps


how about this, can this be used incorporated?




@davidstanton  try the --size-only if you want to sync affected files only , it works. Adding --size-only won't make the entire repo to be synchronised again, but only latest changed files

Like davidstanton likes this

even if file size stays the same and only the contents of the file changed?

i know it doesnt make any sense .. but if the size stays the same and file was changed, then it is going to be "indetified" and copied over s3 .. only that file

Like davidstanton likes this

brilliant, thanks.  Now, any clue how to do the same with a google bucket? lol

sadly I haven't had any experiences with gcp

ok, np.  how about invalidating AWS cloudfront file, based on what 'new" file was uploaded to the SS bucket, rather than at the moment, im invalidating /* (all basically)

this --size-only actually doesn't work, i've just had an instance where the content changed but the size didn't and it didn't upload the file with the new changes

Was there any update to this cause I also did try --size-only and it doesn't work!

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Jira

Admins, notify your Jira instance of system-wide changes with the new admin announcement banner

Hi All! We’re excited to share the launch of an announcement banner that lets Jira site administrators communicate directly to their users across their  Jira Cloud instance.  ...

423 views 13 14
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you