Thread may be stuck, 318 threads are monitored by this Valve and may be stuck

[Raphael]

Hi, All.

I have some issues after I upgrade my Confluence Server from 5.8.14 to 5.10.8.

after upgraded, I shutdown the application, and modified the configuration like "setenv.sh", "server.xml"..

then I start the confluence, i got a lot of stuck in the 'catalina.out' log like below:

12/3/17 2:46 PM) to serve the same request for https://localhost/rest/analytics/1.0/publish/bulk and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 1 thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at java.lang.Object.wait(Native Method)
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1465)

03-Dec-2017 14:47:25.076 WARNING [ContainerBackgroundProcessor[StandardEngine[Standalone]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread "http-nio-8091-exec-1" (id=162) has been active for 69,923 milliseconds (since 12/3/17 2:46 PM) to serve the same request for https://localhost/rest/analytics/1.0/publish/bulk and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 2 thread(s) in total that are monitored by this Valve and may be stuck.

and the number of threads are still increasing, I can't access the application from the web.

I found the request are for these urls:

• https://localhost/rest/analytics/1.0/publish/bulk
• https://localhost/rest/mywork/latest/status/notification/count?_=
• https://localhost/rest/quickreload/latest/16409150?since=
• https://localhost/json/heartbeat.action
• https://localhost/json/startheartbeatactivity.action

I have searched the community, only found one post related to this issue :

Thread may be stuck

But no solution. It may occur in many versions,

Thanks for your watching.

[AnnWorley]

Those messages are just warnings - they are there to bring our attention to threads that may be stuck, so they help when we are troubleshooting a specific issue, but as long as you are not seeing any performance hiccups they are benign. One of your examples is just warning us that a thread took 1.1 minutes. A senior support engineer assured me that these warnings can increase after an upgrade and we don't have to chase them down if Confluence is performing as expected.

If you run into any other post upgrade issues please post them here, we are happy to help.

[Raphael]

Thanks for your attention！

Yes, I am meeting this issue when i upgrade confluence from 5.8.14 to 5.10.8.

at the first time, i also think these logs are just warnings and  we can ignore them  if it does not affect the normal functions. but the end is not what we want. the application hangs here, when we access the web, the warnings increased more .

I hope you can give me some suggestions about how to fix this issue, or some advices about upgrade from 5.8.14 to newest version (This is the exactly need we want.).. 

Thanks a lot in advance.

here is the thread logs:

[AnnWorley]

Please follow Upgrading Confluence to upgrade to the latest version. Since you are already running a version above 5.x, there are no intermediary versions.

Please let me know what the specific symptoms are when Confluence hangs.

[Raphael]

Hi，Ann。

  I think this issue is related to LDAP user synchronization. After I listened to your suggestion, I tried upgraded my application to 6.5.3, the latest version at that time, but still throwing a lot of thread stuck logs in the catalina.out log.

Thread "http-nio-8092-exec-204" (id=438) has been active for 67,139 milliseconds (since 1/9/18 6:34 PM) to serve the same request and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 217 thread(s) in total that are monitored by this Valve and may be stuck.

But this time I found by observing atlassian-confluence.log that there would be the following error in atlassian-confluence.log before a large number of thread stuck errors were thrown:

WARN [http-nio-8092-exec-275] [atlassian.confluence.util.ReadOnlyLicenseCalculator] getNumberOfLicensedUsers Licensing calculation took more than PT2S

After I login into the database, disable the LDAP user directory, the application can be started normally. 

And the problem occurs 2 times in a week,

[atlassian.confluence.util.ReadOnlyLicenseCalculator] getNumberOfLicensedUsers Licensing calculation took more than PT2S

and then Confluence got stuck, the open files (ulimit has been set to 4096) increasing fast, and i have to restart Confluence.

Currently, my solution is to disable LDAP synchronization in the database, and then turn on LDAP again after the application started .

Hope to get your answer. Thank you.

[Ronald Reskowski]

Hi Raphael,

do you have found a solution for this? We are encountering the same issue :(

Thanks in advance for an answer!

Best

Ronald

[David Schmidt]

Hi there, 

I just had a crash as well and googling brought me here.

Any real fix for this? Running 6.14.1 on prem here.

Thanks

David

[Ashok Chelimilla {Appfire}]

Hi @thered

I am facing the same issue Is any real-time fix for it

Thanks,

Ashok Ch

[Shashi Jakku]

Try restarting on a different port by updating conf/server.xml