Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,637,745
Community Members
 
Community Events
196
Community Groups

Bitbucket Pipeline to AWS S3: How to skip unchanged files?

I followed the instructions in https://support.atlassian.com/bitbucket-cloud/docs/deploy-to-aws-with-s3/ to deploy files to AWS S3.

The first step Build artifact creates an artifact folder with say 1000 files. The second step Deploy to S3 uploads the files to S3. I only want to upload the 1 or 2 files that were changed since the last commit to master, not all 1000 files.

atlassian/aws-s3-deploy uses aws s3 sync under the hood: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/sync.html

S3 sync does not have an option to check for differences based on a checksum. A file is uploaded if the timestamp is newer than the timestamp in S3 or if the file size is different. There is an option --size-only, but I can't use that because files might change but still have the same size.

When the repo is checked out in the first step, the timestamp of the files is the current date/time. I can fix the timestamps with git-restore-mtime:

https://manpages.debian.org/unstable/git-restore-mtime/git-restore-mtime.1.en.html

Afterwards each file has the timestamp of the last commit which is what I want.

Problem: when step 1 (Build artifact) passes the artifacts on to step 2 (Deploy to S3), all files are getting a new timestamp (current date/time). As a result, all files are uploaded to S3.

Questions:

* Is it possible to preserve the timestamp of artifact files?

* Is there any other way to upload unchanged files only?

 

1 answer

0 votes
Igor Stoyanov
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
Oct 28, 2022

@Christian Kuenne hi. Thanks for your question.

If you could understand which files are changed, then pass only this files to artifacts (not all).

To verify what files was changed you could try to use Bitbucket API commits.

Regards, Igor

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
PERMISSIONS LEVEL
Site Admin
TAGS
AUG Leaders

Atlassian Community Events