Bitbucket Pipeline to AWS S3: How to skip unchanged files?

[Christian Kuenne]

I followed the instructions in https://support.atlassian.com/bitbucket-cloud/docs/deploy-to-aws-with-s3/ to deploy files to AWS S3.

The first step Build artifact creates an artifact folder with say 1000 files. The second step Deploy to S3 uploads the files to S3. I only want to upload the 1 or 2 files that were changed since the last commit to master, not all 1000 files.

atlassian/aws-s3-deploy uses aws s3 sync under the hood: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/sync.html

S3 sync does not have an option to check for differences based on a checksum. A file is uploaded if the timestamp is newer than the timestamp in S3 or if the file size is different. There is an option --size-only, but I can't use that because files might change but still have the same size.

When the repo is checked out in the first step, the timestamp of the files is the current date/time. I can fix the timestamps with git-restore-mtime:

https://manpages.debian.org/unstable/git-restore-mtime/git-restore-mtime.1.en.html

Afterwards each file has the timestamp of the last commit which is what I want.

Problem: when step 1 (Build artifact) passes the artifacts on to step 2 (Deploy to S3), all files are getting a new timestamp (current date/time). As a result, all files are uploaded to S3.

Questions:

* Is it possible to preserve the timestamp of artifact files?

* Is there any other way to upload unchanged files only?

[Igor Stoyanov]

@Christian Kuenne hi. Thanks for your question.

If you could understand which files are changed, then pass only this files to artifacts (not all).

To verify what files was changed you could try to use Bitbucket API commits.

Regards, Igor