It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Exclude files from upload to s3 bucket using "Amazon s3 object task"

Hi,

I'm trying to replace "aws s3 sync" script task with "Amazon S3 Object" task, on a server install of bamboo  (v6.8.1),  in a build plan that uploads content to a s3 bucket.

 

Trouble i'm facing right now are two:

1) i need to exclude some files from the upload process.

I've tried to get it working using Ant exclude patterns (checking the option "Use Ant patterns to select files"), but with no positive outcome until now. 

 

2) if problem (1) get's solved, and if possible, i would also like to have a 'sync' approach, that is, that the execution of the plan, also removes objects on s3 bucket if they're not present in the local folder anymore.

 

Documentation i've tried following:

https://utoolity.atlassian.net/wiki/spaces/TAWS/pages/19464196/Using+the+Amazon+S3+Object+task+in+Bamboo

http://ant.apache.org/manual/Tasks/defaultexcludes.html

 

HAs anyone got this to work?

 

Thanks

-wm

3 answers

0 votes

Hi Werner,

Are you able to share the ant pattern you’re using, what result you’re expecting and the result you’re getting? This may be enough for the community to help diagnose and suggest a fix.

CCM

Hi CCM,

Sorry, you're right, should have included that information. 

Let's say i have a local directory (from a git cloned on previous task in the plan) with:

./file1.ext

./file2.ext

/dir1/file1-dir1.ext

/dir-dont-upload/file1-dir-dont-upload.ext

./file3.ext
./.some_module_rc

 

My "Amazon S3 Object" task is configured to upload the content of this folder to a s3 bucket.

I've checked the "[ x ]Use Ant patterns to select files" option, and in "Source Local Path" field i've tried a lot of combinations, right now i'm using.

**,<defaultexcludes echo="true" add="/dir-dont-upload/**"/>, 

i've also tried 

**,<exclude add="/dir-dont-upload/**"/>, 

But it always uploads all the files.

If i remove the "**" starting, it doesn't upload anything.

Also, is there a quicker way (like some sandboxes commandline with debug, i could use to test this, as always run the entire plan, is somewhat complicated.

Thanks for any suggestion.

-wm

Hi Werner,

Thank you for using our apps, and sorry for the trouble. I'm afraid your requirements are simply not supported at this point:

1. The 'Use Ant patterns to select files' option currently only supports inclusion of files based on the pattern syntax itself, but not the explicit exclusion of files via any of the 'exclude' XML tags.

  • As a potential workaround, you might want to consider assembling the desired fileset in a new directory, which would allow to use resp. OS level tooling on copy to exclude parts of the source directory hierarchy (specifics depend on the OS and shell in question).
  • I realize that the option's description and linked documentation from Ant are misleading in this regard and we will try to change it so that future users are less likely to run into this, thanks for pointing out this usability flaw.

2. The Amazon S3 Object task does indeed not yet support an equivalent for the aws . s3 . sync action, and we still recommend falling back to the AWS CLI as a workaround for such scenarios accordingly. Please watch and vote for the following story to guide our roadmap:

I hope this helps despite lacking an actual solution.

Cheers,
Steffen

Hi Steffen

 

Thank you for your clarifying answer.

I think that for the time being i'll go back to a shell/system 'aws s3 sync' for this task, at least until UAA-288 is a reality.

 

Have a nice week

-wm

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bamboo

Unable to add or edit Bitbucket Cloud repository in Bamboo

On 31 May, a GDPR-related change went live in the Bitbucket Cloud API that resulted in users not being able to create or edit Bitbucket Cloud Linked repositories in Bamboo. This API update removed t...

695 views 2 7
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you