Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Remote agents get 404 error when attempting to get classpath info from server

We attempted an upgrade of bamboo from v. 5.9.7 to 6.2.5. Server installation was successful, but the remote agents failed to start up. We get the following agent log entries:


INFO | jvm 6 | 2017/12/22 14:47:03 | 2017-12-22 14:47:03,262 INFO [WrapperSimpleAppMain] [ClasspathBuilder] Syncing /1fb/home/build/bamboo-agent-home with https://ci-server/agentServer/bootstrap/classpath/manifest
INFO | jvm 6 | 2017/12/22 14:47:03 | 2017-12-22 14:47:03,276 ERROR [WrapperSimpleAppMain] [ClasspathBuilder] An error has occurred when retrieving classpath data: HTTP/1.1 404 Not Found
INFO | jvm 6 | 2017/12/22 14:47:03 | Exiting due to fatal exception.
INFO | jvm 6 | 2017/12/22 14:47:03 | com.atlassian.bamboo.agent.bootstrap.RemoteAgentHttpException: Response: [HTTP/1.1 404 Not Found] received after manifest retrieval request.
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.terminateWithError(ClasspathBuilder.java:211)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getJarList(ClasspathBuilder.java:200)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.sync(ClasspathBuilder.java:85)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getClassLoader(ClasspathBuilder.java:58)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentContext.newClassLoader(AgentContext.java:329)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentContext.run(AgentContext.java:93)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.run(AgentBootstrap.java:111)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.main(AgentBootstrap.java:50)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO | jvm 6 | 2017/12/22 14:47:03 | at java.lang.reflect.Method.invoke(Method.java:498)
INFO | jvm 6 | 2017/12/22 14:47:03 | at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
INFO | jvm 6 | 2017/12/22 14:47:03 | at java.lang.Thread.run(Thread.java:748)
INFO | jvm 6 | 2017/12/22 14:47:03 | 2017-12-22 14:47:03,277 FATAL [WrapperSimpleAppMain] [AgentBootstrap] Exiting due to fatal exception.
INFO | jvm 6 | 2017/12/22 14:47:03 | com.atlassian.bamboo.agent.bootstrap.RemoteAgentHttpException: Response: [HTTP/1.1 404 Not Found] received after manifest retrieval request.
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.terminateWithError(ClasspathBuilder.java:211)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getJarList(ClasspathBuilder.java:200)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.sync(ClasspathBuilder.java:85)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.ClasspathBuilder.getClassLoader(ClasspathBuilder.java:58)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentContext.newClassLoader(AgentContext.java:329)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentContext.run(AgentContext.java:93)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.run(AgentBootstrap.java:111)
INFO | jvm 6 | 2017/12/22 14:47:03 | at com.atlassian.bamboo.agent.bootstrap.AgentBootstrap.main(AgentBootstrap.java:50)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO | jvm 6 | 2017/12/22 14:47:03 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO | jvm 6 | 2017/12/22 14:47:03 | at java.lang.reflect.Method.invoke(Method.java:498)
INFO | jvm 6 | 2017/12/22 14:47:03 | at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
INFO | jvm 6 | 2017/12/22 14:47:03 | at java.lang.Thread.run(Thread.java:748)
FATAL | wrapper | 2017/12/22 14:47:03 | There were 5 failed launches in a row, each lasting less than 3 seconds. Giving up.
FATAL | wrapper | 2017/12/22 14:47:03 | There may be a configuration problem: please check the logs.
STATUS | wrapper | 2017/12/22 14:47:03 | <-- Wrapper Stopped

 

The agent is trying to reach the server URL https://ci-server/agentServer/bootstrap/classpath/manifest and is receiving a 404 error in return.  

 

Entering the above URL directly into a browser also yields a 404 error as expected.

There are no apparent server log entries at the time of the agent request, or when we entered the URL into a browser manually, but we did find this error on the server log at a previous time, which appears related.

 

2017-12-22 13:50:09,838 ERROR [ajp-nio-8809-exec-30] [FiveOhOh] 500 Exception was thrown.
javax.servlet.ServletException: Error instantiating servlet class com.atlassian.bamboo.agent.classserver.cacheable.CacheableClasspathServlet
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1120)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:828)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:528)
at org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:873)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:670)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.http.entity.ContentType.withCharset(Ljava/nio/charset/Charset;)Lorg/apache/http/entity/ContentType;
at com.atlassian.bamboo.agent.classserver.cacheable.CacheableClasspathServlet.<clinit>(CacheableClasspathServlet.java:56)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:119)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1102)
... 17 more

 

These logs indicate that the JVM couldn't find a method withCharset(Charset c) in the class org.apache.http.entity.ContentType. 

As far as I can tell, the above referenced class (org.apache.http.entity.ContentType) is in httpcore.4.4.6.jar in the WEB-INF/lib folder of the bamboo installation. It appears that this jar file does include a version of a ContentType class with method withCharset(Charset c) that returns a ContentType, according to my decompiler.   Also, there doesn't seem to be any other version of this library in the classpath anywhere.

 

Any thoughts as to how to fix this issue?

 

We have since rolled back to version 5.9.7 but are anxious to get 6.2.5 working due to security vulnerabilities in 5.9.7.

2 answers

0 votes

Hi Peter,

Make sure you have checked the supported platforms.

https://confluence.atlassian.com/display/BAMBOO0602/Supported+platforms

Try a workaround here, based on the same error

https://jira.atlassian.com/browse/BAM-18279

Or you could re-install the remote agent using the 6.2.5 jar

By using verbose classloader logging, we were able to find out where the incorrect version of the org.apache.http.entity.ContentType class came from. It came from bamboo-artifactory-plugin-1.8.2.jar which somehow ended up in atlassian-bamboo/WEB-INF/lib. I am not sure how this jar file ended up in that directory, as it's not in the clean downloaded version of  bamboo 6.2.5 jar file.  But I confirmed this jar file contains an old version of http-core from apache.

 

We renamed this jar file (to prevent its loading into the JVM).  We restarted the server, and this seemed to fix the problem on the server side, or at least we didn't encounter this exception subsequently.

 

However, when we restarted the remote agent, we got a similar exception

 

Constructor threw exception; nested exception is com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: com.google.common.base.Equivalence.identity()Lcom/google/common/base/Equivalence;

 

Again, it seems that the artifactory plugin is the root of the problem.  We found bamboo-artifactory-1.8.2.jar in the agent install location, in the classpath directory.  This jar file also contains an older version of the google guava libraries. 

 

So I think our mystery is solved - an old version of the plugin is the cause of our issues.

Lesson learned - reinstall up-to-date plugins when upgrading Bamboo.

 

FYI - it appears the current version of the Artifactory plugin is 2.3.0. We will try again with this version.

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bamboo

Bamboo 7.1 is here and is packed with value!

I'm happy to announce that Bamboo 7.1 has been released and it’s overflowing with awesome new features. Top-voted issues First and foremost, a bunch of JAC top voted issues has been delivered - y...

587 views 1 6
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you