Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Ability to set max_concurrent_requests for the s3 pipe

When using the S3 pipe https://bitbucket.org/atlassian/aws-s3-deploy/src we need to be able to set the `default.s3.max_concurrent_requests` value as we have lots of little files so the upload is slow. 

 

The command to set it is `aws configure set default.s3.max_concurrent_requests 20` It would be great if this could be a configuration value. The default is 10.

1 answer

@Galyna Zholtkevych Not sure if you are the right person to talk to about this. I would love to help and provide a PR if that is possible. 

@Andrew McClenaghan I am the right person :)

Thanks for raising this. Soon we are going to work on s3 to improve it and can think about additional parameters. I will include your question to the discussion and notify you once we release a new update.

Regards, Galyna

@Andrew McClenaghan also , if you want to make a contribution, feel free to do it. We are happy to see new contributors. Just follow contributing guide in the pipe and don't forget to make a changeset in the pr, that will provide pipe's version bump on merge.

Also, take into account that we are going to refactor this pipe to Python in the next update, so if you do it now, you anyway will have to rewrite this to Python.

Generally, me , I like the idea if we pass additional aws config parameters and configure these params in the pipe.

I note that it would be different from any extra parameters types, because it provides params for aws config file, but not params to pass to functions variables.

But I would discuss this within the team.

If you want to do the contribution, tell us, so we can either wait for your contribution or implement this feature ourselves.

Regards, Galyna

How far off do you think the change to python will be? We are having a very slow deploy at the moment so just comparing the time to fix this in the cli version vs waiting for the python version.

@Andrew McClenaghan refactoring to Python in our infrastructure is caused by other reasons, but python is much more convenient for different parameters and config things, since we can allow more flexible data structures for variables to pass in the pipe. Like this you'd have the natural ability to make proper config for you. And since it is natural, it should work faster because of it is natural and you don't have to use workarounds like this.

Regards, Galyna

Hi @Galyna Zholtkevych 

 

Thanks for that. 

 

Are you able to provide a timeline for this change? I am trying to work out if I need to do the PR on the current version or if we can wait for the move to Python.

 

Cheers,
Andrew

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket Pipelines

Bitbucket Pipelines Runners is now in open beta

We are excited to announce the open beta program for self-hosted runners. Bitbucket Pipelines Runners is available to everyone. Please try it and let us know your feedback. If you have any issue...

2,394 views 51 18
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you