Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

AWS Spot instance create failed with Invalid BASE64 encoding of user data

Hey, Atlassian Team

We starting to hit Spot Instance create failure with Invalid BASE64 encoding of user data since 2AM Nov 5th 2019 GMT time. 
We are using Bamboo 6.7.2 build 60706. 

We were hitting random issue like this before, maybe 2 out of 100 times, but we worked around it by store most of the userdata into S3 bucket. 

Since this morning, when we enable Spot Instance Support in Bamboo, we will start hit this error 100% time. This feels like some AWS side change. Do you have any suggestion? 

 

We noticed this JIRA https://jira.atlassian.com/browse/BAM-19945?focusedCommentId=1944640&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel

 

This JIRA is saying deliver a fix in Bamboo 6.9. https://jira.atlassian.com/browse/BAM-19945?focusedCommentId=1944640&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel

 

Will update to 6.9 gives back the ability to use Spot Agent for bamboo? 

 

 

 

 

Errors logs

System Error Details
Elastic Bamboo Error : EC2 Instance failed to start: EC2 instance order for image ami-XXXXXXX failed.
(com.amazonaws.services.ec2.model.AmazonEC2Exception : Invalid BASE64 encoding of user data (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: ab6ad633-d10d-4a79-a725-e9e5c2049af1))

Occurred: 05 Nov 2019, 6:54:33 AM
com.amazonaws.services.ec2.model.AmazonEC2Exception: Invalid BASE64 encoding of user data (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: ab6ad633-d10d-4a79-a725-e9e5c2049af1)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1658)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1322)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1072)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:17597)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:17566)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:17555)
at com.amazonaws.services.ec2.AmazonEC2Client.executeRequestSpotInstances(AmazonEC2Client.java:16281)
at com.amazonaws.services.ec2.AmazonEC2Client.requestSpotInstances(AmazonEC2Client.java:16252)
at sun.reflect.GeneratedMethodAccessor3136.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.atlassian.aws.CallTimingProxy$1.invoke(CallTimingProxy.java:34)
at com.sun.proxy.$Proxy1639.requestSpotInstances(Unknown Source)
at com.atlassian.aws.ec2.awssdk.launch.AwsSpotInstanceLauncher.call(AwsSpotInstanceLauncher.java:77)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.launchInstance(RemoteEC2InstanceImpl.java:363)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.backgroundStart(RemoteEC2InstanceImpl.java:346)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.access$100(RemoteEC2InstanceImpl.java:39)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$1.run(RemoteEC2InstanceImpl.java:95)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$CatchingRunnableDecorator.run(RemoteEC2InstanceImpl.java:79)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)




Nov 5, 2019 6:54:33 AM Requested that new elastic instance be created for configuration: Quick Test / ami-XXXXXXX
Nov 5, 2019 6:54:33 AM Detected that a pending instance request for image [Quick Test], ami: ami-XXXXXXX has been abandoned.

 

 

 

1 answer

1 accepted

2 votes
Answer accepted

Hi @Lei P

I saw you opened a ticket with our support team for this issue and it was already resolved.
I'm adding the findings here in case other users face the same issue.

Problem

This problem may happen eventually. The issue is caused by the following bug which was fixed in Bamboo 6.9.0:

Resolution

Upgrade the Bamboo instance to 6.9.0 or any newer version.

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bamboo

Bamboo 101 Video

G’day Community! As we gear up to introduce Bamboo Data Center to the world, we wanted to make sure that we shared a bit more about Bamboo, the product. Our team has put together an overview video ...

223 views 4 6
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you