Thread may be stuck, 318 threads are monitored by this Valve and may be stuck

Hi, All.

I have some issues after I upgrade my Confluence Server from 5.8.14 to 5.10.8.

after upgraded, I shutdown the application, and modified the configuration like "setenv.sh", "server.xml"..

then I start the confluence, i got a lot of stuck in the 'catalina.out' log like below:

 

12/3/17 2:46 PM) to serve the same request for https://localhost/rest/analytics/1.0/publish/bulk and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 1 thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at java.lang.Object.wait(Native Method)
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1465)

 

03-Dec-2017 14:47:25.076 WARNING [ContainerBackgroundProcessor[StandardEngine[Standalone]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread "http-nio-8091-exec-1" (id=162) has been active for 69,923 milliseconds (since 12/3/17 2:46 PM) to serve the same request for https://localhost/rest/analytics/1.0/publish/bulk and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 2 thread(s) in total that are monitored by this Valve and may be stuck.

 

and the number of threads are still increasing, I can't access the application from the web.

I found the request are for these urls:

  • https://localhost/rest/analytics/1.0/publish/bulk
  • https://localhost/rest/mywork/latest/status/notification/count?_=
  • https://localhost/rest/quickreload/latest/16409150?since=
  • https://localhost/json/heartbeat.action
  • https://localhost/json/startheartbeatactivity.action

I have searched the community, only found one post related to this issue :

Thread may be stuck

But no solution. It may occur in many versions,

Thanks for your watching.

1 answer

0 votes
Ann Worley Atlassian Team Dec 05, 2017

Those messages are just warnings - they are there to bring our attention to threads that may be stuck, so they help when we are troubleshooting a specific issue, but as long as you are not seeing any performance hiccups they are benign. One of your examples is just warning us that a thread took 1.1 minutes. A senior support engineer assured me that these warnings can increase after an upgrade and we don't have to chase them down if Confluence is performing as expected.

If you run into any other post upgrade issues please post them here, we are happy to help.

Thanks for your attention!

Yes, I am meeting this issue when i upgrade confluence from 5.8.14 to 5.10.8.

at the first time, i also think these logs are just warnings and  we can ignore them  if it does not affect the normal functions. but the end is not what we want. the application hangs here, when we access the web, the warnings increased more .

I hope you can give me some suggestions about how to fix this issue, or some advices about upgrade from 5.8.14 to newest version (This is the exactly need we want.).. 

Thanks a lot in advance.

here is the thread logs:

theads_hangs.png

Ann Worley Atlassian Team Dec 06, 2017

Please follow Upgrading Confluence to upgrade to the latest version. Since you are already running a version above 5.x, there are no intermediary versions.

Please let me know what the specific symptoms are when Confluence hangs.

Hi,Ann。

  I think this issue is related to LDAP user synchronization. After I listened to your suggestion, I tried upgraded my application to 6.5.3, the latest version at that time, but still throwing a lot of thread stuck logs in the catalina.out log.

 

Thread "http-nio-8092-exec-204" (id=438) has been active for 67,139 milliseconds (since 1/9/18 6:34 PM) to serve the same request and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 217 thread(s) in total that are monitored by this Valve and may be stuck.

 

But this time I found by observing atlassian-confluence.log that there would be the following error in atlassian-confluence.log before a large number of thread stuck errors were thrown:

 

WARN [http-nio-8092-exec-275] [atlassian.confluence.util.ReadOnlyLicenseCalculator] getNumberOfLicensedUsers Licensing calculation took more than PT2S

 

After I login into the database, disable the LDAP user directory, the application can be started normally. 

And the problem occurs 2 times in a week,

[atlassian.confluence.util.ReadOnlyLicenseCalculator] getNumberOfLicensedUsers Licensing calculation took more than PT2S

and then Confluence got stuck, the open files (ulimit has been set to 4096) increasing fast, and i have to restart Confluence.

Currently, my solution is to disable LDAP synchronization in the database, and then turn on LDAP again after the application started .

 

Hope to get your answer. Thank you.

Hi Raphael,

do you have found a solution for this? We are encountering the same issue :(

Thanks in advance for an answer!

Best

Ronald

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted Thursday in Confluence

What are your project planning tips?

Hello Community,  Jessica here from the Confluence product marketing team! Today I wanted to get your takes on project planning –– what works, what doesn’t, how do you know if you’re doing it r...

155 views 0 3
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you