When using the S3 pipe https://bitbucket.org/atlassian/aws-s3-deploy/src we need to be able to set the `default.s3.max_concurrent_requests` value as we have lots of little files so the upload is slow.
The command to set it is `aws configure set default.s3.max_concurrent_requests 20` It would be great if this could be a configuration value. The default is 10.
@Andrew McClenaghan I am the right person :)
Thanks for raising this. Soon we are going to work on s3 to improve it and can think about additional parameters. I will include your question to the discussion and notify you once we release a new update.
@Andrew McClenaghan also , if you want to make a contribution, feel free to do it. We are happy to see new contributors. Just follow contributing guide in the pipe and don't forget to make a changeset in the pr, that will provide pipe's version bump on merge.
Also, take into account that we are going to refactor this pipe to Python in the next update, so if you do it now, you anyway will have to rewrite this to Python.
Generally, me , I like the idea if we pass additional aws config parameters and configure these params in the pipe.
I note that it would be different from any extra parameters types, because it provides params for aws config file, but not params to pass to functions variables.
But I would discuss this within the team.
If you want to do the contribution, tell us, so we can either wait for your contribution or implement this feature ourselves.
@Andrew McClenaghan refactoring to Python in our infrastructure is caused by other reasons, but python is much more convenient for different parameters and config things, since we can allow more flexible data structures for variables to pass in the pipe. Like this you'd have the natural ability to make proper config for you. And since it is natural, it should work faster because of it is natural and you don't have to use workarounds like this.
We are excited to announce the open beta program for self-hosted runners. Bitbucket Pipelines Runners is available to everyone. Please try it and let us know your feedback. If you have any issue...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events