How do I push only the changed files to S3?

[Mike Green]

So when files are being edit or changes are made and edit in a bitbucket repository, how can I push only the changes, is there a way or pipeline to do so?

[lissyaka]

Mike Green, hi. It's a good case to use pipe: aws-s3-deploy, which under the hood run: aws s3 sync command.

Try this in your pipeline configuration:

script:
  - pipe: atlassian/aws-s3-deploy:1.1.0
    variables:
      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
      AWS_DEFAULT_REGION: 'us-east-1'
      S3_BUCKET: 'my-bucket-name'
      LOCAL_PATH: 'build'

Cheers.

[Mike Green]

This pipe only push all the files to S3 bucket, but we only want to push the changed files only to the S3 bucket. Is there a way to do that?

[lissyaka]

According to docs i provided previously: aws s3 sync recursively copies new and updated files from the source directory to the destination.

This pipe logic:

run aws s3 sync ${LOCAL_PATH} s3://${S3_BUCKET}/ ${ARGS_STRING[@]} ${AWS_DEBUG_ARGS}

[Quang Lai]

That may be true, but when using it in Bitbucket Pipelines, the files don't retain their date/time from the code repo.  The file always gets the current date/time, and so AWS S3 sync will upload *all* the files (because all the files have a newer file timestamp than what's on S3).  This is easily tested doing a "ls -laR" in the pipeline and running the pipeline more than once, a few minutes apart.

So what we need is a way for Bitbucket Pipelines to preserve the files' date/time, instead of always reflecting the current date/time.