nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

bamboo elastic agent permission denied on S3 sync

Jeff Behl

Seen a number of posts about this sort of issue...don't get why it seems to happen regularly. Atlassian - looks like something you need to fix? from us-west-1

[root@ip-10-35-12-70 bamboo]# /opt/bamboo-elastic-agent/bin/bamboo-elastic-agent
Syncing Elastic Bamboo Agent files...
2015-12-13 04:30:22,812 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 04:30:24,521 INFO [main] [S3Synchroniser] Syncing s3://bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 04:30:24,536 INFO [main] [AmazonClients] Detecting bucket location for [bamboo-agent-release-us-w1]
2015-12-13 04:30:25,812 WARN [main] [AmazonClients] Unable to get bucket location for [bamboo-agent-release-us-w1], using default. Error: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 0551DDC8AC66E08A), S3 Extended Request ID: vufOZs1Jw7DKEKq2bwAT6bnE/ZUPQFdI+7jtK7ITL9jLTCuV9GhBZwUXvPa6Q8TCLrSUvhc+7lc=
2015-12-13 04:30:25,812 INFO [main] [S3Synchroniser] Fetching the list of remote objects...
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: B8E6EC6AF92964D7), S3 Extended Request ID: sic01W4ilb9JpvxXGb/hpsM6oBtqNjc7fmRwCildB9PLnKgMYHJvOvwviiuyySv0Pxo0E8+K0fA=
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3604)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3557)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:647)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:632)
	at com.atlassian.aws.s3.S3Synchroniser.getObjectNamesAndHashes(S3Synchroniser.java:341)
	at com.atlassian.aws.s3.S3Synchroniser.sync(S3Synchroniser.java:163)
	at com.atlassian.bamboo.agent.elastic.S3Sync.sync(S3Sync.java:72)
	at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.install(ElasticAgentInstaller.java:76)
	at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:173)

Accepted answers

Jeff Behl

So here's the answer: Atlassian's S3 bucket that the elastic agent uses to sync some jars and whatnot is restricted to EC2 IP addresses. Our VPC traffic runs through a VPN link and exits through our data center, meaning the source appeared to be non-EC2. The workaround was to enable EC2 VPN endpoints which allows for internal VPC hosts to access S3 directly:

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html

All comments

Przemek Bruski

Is it still happening?

Jeff Behl

yep...just tried: 2015-12-13 19:23:07,125 WARN [main] [AmazonClients] Unable to get bucket location for [bamboo-agent-release-us-w1], using default. Error: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1061719D3D6F6E92), S3 Extended Request ID: 1V4qvBwT0neOf+Xtd7avXigpx5XkvO4et33OVQtCEzLpcJFjOQB8+YfNXoLjM1uAtEl5in8yHII= 2015-12-13 19:23:07,125 INFO [main] [S3Synchroniser] Fetching the list of remote objects... Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 5CA1ECA6CA63A5FF), S3 Extended Request ID: 00pQkCl5ITT5AGCqrQ1krQInpbXh4ofnc7R5XWxoWk6bR3MXAwoYK4cRhdnW/rZ25+GiatrvcO0=

Jeff Behl

@Przemyslaw Bruski - here's the full (just tried). And I should have mentioned this earlier: this is a custom image that I've created to do this. The instance was started by bamboo, however, but sits there at the "Pending" state from the Bamboo UI. The below is after I've logged into this instance to see what could have gone wrong and tried running the bamboo-elastic-agent manually. It's also in the "/home/bamboo/bamboo-elastic-agent.out" file, so I'm assuming it's running correctly, just getting that error and failing..

[root@ip-10-35-12-72 bin]# ./bamboo-elastic-agent
Syncing Elastic Bamboo Agent files...
2015-12-13 19:23:04,364 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 19:23:05,885 INFO [main] [S3Synchroniser] Syncing s3://bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 19:23:05,886 INFO [main] [AmazonClients] Detecting bucket location for [bamboo-agent-release-us-w1]
2015-12-13 19:23:07,125 WARN [main] [AmazonClients] Unable to get bucket location for [bamboo-agent-release-us-w1], using default. Error: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1061719D3D6F6E92), S3 Extended Request ID: 1V4qvBwT0neOf+Xtd7avXigpx5XkvO4et33OVQtCEzLpcJFjOQB8+YfNXoLjM1uAtEl5in8yHII=
2015-12-13 19:23:07,125 INFO [main] [S3Synchroniser] Fetching the list of remote objects...
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 5CA1ECA6CA63A5FF), S3 Extended Request ID: 00pQkCl5ITT5AGCqrQ1krQInpbXh4ofnc7R5XWxoWk6bR3MXAwoYK4cRhdnW/rZ25+GiatrvcO0=
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
..
..

Jeff Behl

Apologies with the conversation with myself, but a few more data points...

The problem seems to reside round the detection of the region, perhaps due to the custom image running from inside of a VPC? Running a stock Ubuntu image loads fine:

2015-12-13 20:44:47,248 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 20:44:51,092 INFO [main] [S3Utils] Syncing s3://bamboo-agent-release-us-w1/5.9.7/b3f798e03f020d72f10564280b47840ba203ae32/ to /opt/bamboo-elastic-agent
2015-12-13 20:44:51,092 INFO [main] [AmazonClients] Detecting bucket location for [bamboo-agent-release-us-w1]
2015-12-13 20:44:53,079 INFO [main] [AmazonClients] Set S3 endpoint to: s3-us-west-1.amazonaws.com
2015-12-13 20:44:53,079 INFO [main] [S3Utils] Fetching the list of remote objects...

It should be noted it is running outside of a VPC and has a public address.

The custom image I've created and started via bamboo resides inside of a VPC and does not have a public IP address. It is able to talk to the internet, though through a VPN connection. It difference seems to be:

2015-12-13 04:30:25,812 WARN [main] [AmazonClients] Unable to get bucket location for [bamboo-agent-release-us-w1], using default.

Any way I can override the S3 endpoint? Or influence the "detecting bucket location" ?

Jeff Behl

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

bamboo

bamboo-administration

aws

amazon