For quite some time my JIRA has been experiencing rapid increase in active threads count. JavaMelody shows it like this:
Usually it goes on and on until I have to manually restart JIRA. At the time of active threads increase some issues hang up and it is not possible to open issue pages of hang up issues. I tried to run garbage collector, it does not help.
In the past this situation with rapid increase of active threads happened when background indexing was started and when Bulk Clone started. This time neither of those things happen and I would still experience problem with active threads for some reason.
Could please somebody recommend some tweak to resolve this problem? I would appreciate any help.
Take several thread dumps with several seconds interval. Analyze the stack traces - if there are leaking threads, you'll see many threads with the same stack trace. Looking at the frames, you should be able to pinpoint the faulty add-on or functionality in JIRA that's causing this.
I have been able to find threads with the same stack trace:
"StreamsCompletionService::thread-989" #12299 prio=5 os_prio=0 tid=0x0000000025ca7000 nid=0x1d18 waiting on condition [0x000000006188f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000005e0f3c0d0> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Locked ownable synchronizers: - None
But I don't know how to map it to the faulty plugin functionality as it doesn't give any information about source of the trace. Could you please recommend me how to do this? Thanks.
You definitely need to take the thread dumps when you see your thread count rising.
That looks like an idling thread from a thread pool. Look for threads with bigger stack traces.
It is normal to see an increase in the number of threads when usage increases - the web server will create more worker threads to handle web requests. I've not used JavaMelody before, but if "active threads" stands for number of threads handling HTTP requests in Tomcat (as their documentation seems to explain) then you should take a tread dump at the peak of usage and look at it. Also check the CPU and memory usage - the reason for high number of active threads could be resource starvation on the server.
I have multiple projects that use variations of the same base workflow. The variations depend on the requirements of the project or issue type. The variations mostly come in the form of new statuses ...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events