We have been running Bamboo version 6.3.1 server. All of sudden when we try and start up, it seems to start coming up but then shuts itself down. It's not obvious what the cause for the shutdown is from the logs, we are hoping someone can take a look at the log and give us a hint. Here is the content of the catalina.out, last 19k characters:
25-Jan-2018 06:13:44.332 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [1-PullRequestDetectionBackgroundThread:pool-4-thread-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.atlassian.bamboo.plan.ItemDetections.getDetectionRequest(ItemDetections.java:36)
com.atlassian.bamboo.plan.pullrequest.PullRequestDetector$1.run(PullRequestDetector.java:65)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:106)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:99)
com.atlassian.bamboo.plugin.BambooPluginUtils.callUnsafeCode(BambooPluginUtils.java:53)
com.atlassian.bamboo.plan.pullrequest.PullRequestDetector.run(PullRequestDetector.java:164)
com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.333 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [lucene-tracked-searchers-pruner-pool-5-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.333 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [lucene-tracked-searchers-pruner-pool-7-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.333 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [lucene-tracked-searchers-pruner-pool-9-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.333 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [mail-queue-processor-pool-14-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
com.atlassian.bamboo.mail.EmailService$MailQueueProcessor.run(EmailService.java:71)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.334 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [XMPP Worker] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.atlassian.bamboo.util.concurrent.BlockingCircularFifoQueue.waitForItem(BlockingCircularFifoQueue.java:37)
com.atlassian.bamboo.util.concurrent.BlockingCircularFifoQueue.take(BlockingCircularFifoQueue.java:22)
com.atlassian.bamboo.xmpp.BambooSmackClient$XmppWorkerThread.run(BambooSmackClient.java:251)
25-Jan-2018 06:13:44.334 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [12-BranchDetectionBackgroundThread:pool-20-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.atlassian.bamboo.plan.ItemDetections.getDetectionRequest(ItemDetections.java:36)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector$1.run(BranchDetectionServiceImpl.java:119)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:106)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:99)
com.atlassian.bamboo.plugin.BambooPluginUtils.callUnsafeCode(BambooPluginUtils.java:53)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector.run(BranchDetectionServiceImpl.java:168)
com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.334 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [12-BranchDetectionBackgroundThread:pool-20-thread-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.atlassian.bamboo.plan.ItemDetections.getDetectionRequest(ItemDetections.java:36)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector$1.run(BranchDetectionServiceImpl.java:119)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:106)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:99)
com.atlassian.bamboo.plugin.BambooPluginUtils.callUnsafeCode(BambooPluginUtils.java:53)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector.run(BranchDetectionServiceImpl.java:168)
com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.335 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [12-BranchDetectionBackgroundThread:pool-20-thread-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.atlassian.bamboo.plan.ItemDetections.getDetectionRequest(ItemDetections.java:36)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector$1.run(BranchDetectionServiceImpl.java:119)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:106)
com.atlassian.bamboo.plugin.BambooPluginUtils$Runnable.call(BambooPluginUtils.java:99)
com.atlassian.bamboo.plugin.BambooPluginUtils.callUnsafeCode(BambooPluginUtils.java:53)
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl$BranchDetector.run(BranchDetectionServiceImpl.java:168)
com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
java.lang.Thread.run(Thread.java:748)
25-Jan-2018 06:13:44.335 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [LastResortTerminationThread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Thread.sleep(Native Method)
java.lang.Thread.sleep(Thread.java:340)
java.util.concurrent.TimeUnit.sleep(TimeUnit.java:386)
com.atlassian.bamboo.container.ShutdownEnforcer$1.run(ShutdownEnforcer.java:26)
25-Jan-2018 06:13:44.336 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.336 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.336 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.336 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.337 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.337 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@10da5bff]) and a value of type [org.acegisecurity.context.SecurityContextImpl] (value [org.acegisecurity.context.SecurityContextImpl@caf53fa: Authentication: org.acegisecurity.adapters.PrincipalAcegiUserToken@caf53fa: Username: SYSTEM; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: ROLE_SYSTEM, ROLE_ADMIN]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
25-Jan-2018 06:13:44.344 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8085"]
25-Jan-2018 06:13:44.347 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 34476 ms
2018-01-25 06:13:59,228 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[main,5,main] is still running.
2018-01-25 06:13:59,229 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[lucene-tracked-searchers-pruner-pool-2-thread-1,5,main] is still running.
2018-01-25 06:13:59,229 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[lucene-tracked-searchers-pruner-pool-5-thread-1,5,main] is still running.
2018-01-25 06:13:59,229 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[lucene-tracked-searchers-pruner-pool-7-thread-1,5,main] is still running.
2018-01-25 06:13:59,230 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[lucene-tracked-searchers-pruner-pool-9-thread-1,5,main] is still running.
2018-01-25 06:13:59,230 WARN [LastResortTerminationThread] [ShutdownEnforcer] Shutting down Bamboo despite the fact that a non-daemon thread Thread[mail-queue-processor-pool-14-thread-1,5,main] is still running.
2018-01-25 06:13:59,230 INFO [LastResortTerminationThread] [lifecycle] *******************************
2018-01-25 06:13:59,230 INFO [LastResortTerminationThread] [lifecycle] * Bamboo has been shut down *
2018-01-25 06:13:59,230 INFO [LastResortTerminationThread] [lifecycle] *******************************
2018-01-25 06:13:59,230 WARN [LastResortTerminationThread] [ShutdownEnforcer] Calling System.exit()
25-Jan-2018 06:13:59.232 INFO [Thread-28] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8085"]
25-Jan-2018 06:13:59.284 INFO [Thread-28] org.apache.catalina.core.StandardService.stopInternal Stopping service Catalina
25-Jan-2018 06:13:59.288 INFO [Thread-28] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8085"]
25-Jan-2018 06:13:59.291 INFO [Thread-28] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8085"]
