Elastic agent "Wire format negotiation timeout"

[Caleb Anderson]

I cannot get my Elastic instances to connect to my Bamboo server.

• We are using AMI: ami-32db8458 – I believe this is your default Ubuntu AMI.
• The Instance starts in AWS but the Agent Status remains in the "Pending" status.
• We can SSH from the Bamboo server to the agent.
• We can SSH from the agent to the bamboo server.
• We can communicate to the Bamboo server on ports 46593, 22, 4527, and 26224.
• We can communicate to the agent on those same ports.
• Java version is "1.8.0_65" (checked per this answer: https://answers.atlassian.com/questions/25139249).
• I have added both servers to the other's hosts file per https://answers.atlassian.com/questions/231544 and that didn't make any difference.
  • I added it with all possible names that it may be using when trying to call to the other. 
• As mentioned above, we have tested port 26224 per Elastic Agent stuck "Pending" on Custom Elastic Image.
• I can't seem to find what Bruno Rosa is referring to in his answer to https://answers.atlassian.com/questions/26643836.  We don't have anything in the Bamboo server config referring to port 4527.

It might be related to https://answers.atlassian.com/questions/265786 because the Bamboo server and the agents are in different AWS spaces and we want them to communicate across private IP addresses and "Enable elastic IP management" doesn't seem to be an option anymore.

The ending of the bamboo-elastic-agent.out is this:

2016-05-11 15:43:48,748 INFO [AgentRunnerThread] [QuartzScheduler] Scheduler scheduler_$_NON_CLUSTERED started.
2016-05-11 15:43:48,972 WARN [AgentRunnerThread] [PropertyPlaceholderConfigurer] Could not load properties from class path resource [bamboo-init.properties]: class path resource [bamboo-init.properties] cannot be opened because it does not exist
2016-05-11 15:43:49,035 INFO [AgentRunnerThread] [ElasticAgentRegistrationBean] Current agent remote definition: Elastic Agent on i-ef16d975 Elastic agent on instance i-ef16d975, configuration: Test EB Agent
2016-05-11 15:43:59,698 WARN [AgentRunnerThread] [FailoverTransport] Transport (tcp://127.0.0.1:4527) failed, reason:  java.io.IOException: Wire format negotiation timeout: peer did not send his wire format., attempting to automatically reconnect
2016-05-11 15:54:31,554 ERROR [ActiveMQ Task-2] [FailoverTransport] Failed to connect to [tcp://localhost:4527?wireFormat.maxInactivityDuration=300000] after: 10 attempt(s)
2016-05-11 15:54:31,578 INFO [ActiveMQ Connection Executor: unconnected] [PooledConnectionFactory] Expiring connection ActiveMQConnection {id=ID:ip-10-10-34-44-58092-1462981429198-1:1,clientId=ID:ip-10-10-34-44-58092-1462981429198-0:1,started=false} on IOException: java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
2016-05-11 15:54:31,581 ERROR [AgentRunnerThread] [ElasticAgentRegistrationBean] org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Wire format negotiation timeout: peer did not send his wire format.
Exiting due to fatal exception.
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:30)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean': Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Wire format negotiation timeout: peer did not send his wire format.
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:407)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1545)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:267)
	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:77)
	at com.atlassian.bamboo.agent.elastic.client.ElasticAgent.start(ElasticAgent.java:31)
	... 6 more
Caused by: org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.registrationQueue]; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Wire format negotiation timeout: peer did not send his wire format.
	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.convertJmsInvokerAccessException(JmsInvokerClientInterceptor.java:433)
	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:206)

and then repeats that trace a few more time.

[Przemek Bruski · Accepted solution]

Is the Bamboo base url resolveable from the Bamboo server host?

[Caleb Anderson]

Yes. When I curl "[[base_url]]/about.action" I get the HTML Content of the page.

[Caleb Anderson]

although the base url resolved, the hostname (ip-10-...) didn't.  I added that to the hosts file of the Bamboo server and it seems to have resolved my connection!