New Elastic Bamboo agent stuck pending - AmazonS3Exception in log

Hi Team,

I've just setup a new custom elastic windows image on amazon EC2.

The connections are working ok and the image has been loaded onto a server, but when I start the agent from within bamboo, the elastic agent gets stuck on "pending".

I've logged onto the instance to take a look at the bamboo-elastic-agent.out log file, and can see the following error. Do I need to open an additional port?

JAVA_HOME: C:\Program Files\Java\jdk1.7.0_40 
PATH: C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Amazon\cfn-bootstrap\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Git\cmd 
HOME: \Users\Bamboo 
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
2013-09-19 17:13:56,773 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-us-w2/5.0.2-OD-2/a564255d62b4cb45e6b8e2acebe7358660f32b30/ to /opt/bamboo-elastic-agent
2013-09-19 17:14:02,217 INFO [main] [S3Utils] Syncing s3://bamboo-agent-release-us-w2/5.0.2-OD-2/a564255d62b4cb45e6b8e2acebe7358660f32b30/ to \opt\bamboo-elastic-agent
2013-09-19 17:14:02,217 INFO [main] [S3Utils] Fetching the list of remote objects...
Exception in thread "main" AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: 0CB2B4F164F533B0, AWS Error Code: AccessDenied, AWS Error Message: Access Denied, S3 Extended Request ID: wKkYmQcytOOmP2p5hQC7uiXeNBDZsUPdbFa2VAH3GXUg/+AOxPkpIppcQxSgNukX
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:644)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:338)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:190)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2974)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2945)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:478)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:462)
	at com.atlassian.aws.s3.S3Utils.getObjectNamesAndHashes(S3Utils.java:331)
	at com.atlassian.aws.s3.S3Utils.sync(S3Utils.java:159)
	at com.atlassian.bamboo.agent.elastic.S3Sync.sync(S3Sync.java:72)
	at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:66)

4 answers

0 votes

I've refreshed the bucket policies, can you try again?

Thanks - I'm now getting an error from within Bamboo on Demand itself when starting the instance - so I can't yet determine if your change has fixed my issue.

I'm getting "Failed to start elastic instances: Error when starting a new instance".

The System errors page in Bamboo contains no errors. Would this be in the log? How do I get at the Bamboo log using On Demand?

Please contact support, they'll get the log and probably get back to devs to improve that error message.

"failed to start elastic instances error when starting a new instance"

Am getting this error.. please help me

I'm getting the same error message.

bamboo-elastic-agent.log

JAVA_HOME: C:\Program Files\Java\jdk1.7.0_40 
PATH: C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Amazon\cfn-bootstrap\;C:\Program Files (x86)\Git\cmd;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\NuGet;%JAVA_HOME\bin% 
HOME: \Users\Bamboo 
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
2013-09-30 00:44:47,767 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-ap-se2/5.2-OD-2/fa5fd0da044fad69f7648355dce5c5dc8f613e65/ to /opt/bamboo-elastic-agent
2013-09-30 00:44:50,903 INFO [main] [S3Utils] Syncing s3://bamboo-agent-release-ap-se2/5.2-OD-2/fa5fd0da044fad69f7648355dce5c5dc8f613e65/ to \opt\bamboo-elastic-agent
2013-09-30 00:44:50,903 INFO [main] [S3Utils] Fetching the list of remote objects...
Exception in thread "main" AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: F61A967F320E768F, AWS Error Code: AccessDenied, AWS Error Message: Access Denied, S3 Extended Request ID: u2Tq0QiuNyYayZKX5ncnnNLMj7xU4j3XnlVN9kNVNu3lpk/pjtbn+u1ptVvqXy51
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:644)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:338)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:190)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2974)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2945)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:478)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:462)
	at com.atlassian.aws.s3.S3Utils.getObjectNamesAndHashes(S3Utils.java:331)
	at com.atlassian.aws.s3.S3Utils.sync(S3Utils.java:159)
	at com.atlassian.bamboo.agent.elastic.S3Sync.sync(S3Sync.java:72)
	at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:66)

I raised a bug for it https://jira.atlassian.com/browse/BAM-13826

I'm only guessing but is it that the automated deployments of the new agents aren't setting the right permissions on the s3 buckets?

This has just started to happen for us.

Do you need to do another refresh?

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Monday in Jira Ops

Jira Ops Early Access Program Update #1: Announcing our next feature and a new integration

Thanks for signing up for Jira Ops! I’m Matt Ryall, leader for the Jira Ops product team at Atlassian. Since this is a brand new product, we’ll be delivering improvements quickly and sharing updates...

347 views 0 8
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you