Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Next challenges

Recent achievements

Recognition

  • Give kudos
  • My kudos

Leaderboard

  • Global

Trophy case

Kudos (beta program)

Kudos logo

You've been invited into the Kudos (beta program) private group. Chat with others in the program, or give feedback to Atlassian.

View group

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Jira usage very slow and cpu-usage very high, after a couple of days Edited

Hi there,

if this problem occures, we try to restart the jira-service first. But its never successful. The jira-service goes into the state "stopping service" but it could/will not be stopped within ~20-30min (very long time).

image.png

It`s necessary to terminate the corresponding tomcat-process manual.

Afterwards the jira-service is stoppend and the cpu-usage is in the normal range again ~10%. 

image.png

The problem repeats again after a couple of days.

 

We are running jira v8.2.4 under WindowsServer2012 R2 with an oracle database 12.1.0.2.0.

Our jira-instance is only accessible from our intranet. The instance is not reachable from the internet.

We created a thread dump from the last occurrence.

 

https://www.dropbox.com/s/bw1glftxe1pcwha/jira_threaddump.txt?dl=0

 

Can anybody help us what might cause this problem?

Thanks in advance for any help.

 

Regards 

-Knut

 

3 answers

0 votes
Andy Heinzer Atlassian Team Jan 14, 2020

Hi Knut,

I understand that your Jira instance appears to be getting into an unresponsive state after a few days and you are looking for more information about why.  Thanks for including the thread dump.  I used the samurai.jar utility to take a closer look at this thread dump.  I found that there were at least 2 threads blocked, and another 2 that were actively blocking.

Having blocked threads in that file are a clear indicator of a performance problem here, but by themselves they typically are not the source of the problem.  For that, we typically need to look at the blocking threads instead, which is where samurai is helpful.  The two threads that appear to be causing this blocking were titled:

Structure-Jobs4cb20abe System-Thread#2
JIRA-INFORM-Thread-0

The first thread is clearly related to the Structure plugin.  It might help to try to reach out to the Structure support team for help with troubleshooting that plugin to Jira.   I would try to focus on that thread first.

The second thread appears to be related to the Jira's native event listeners.  These can be used for triggering notifications on changes, or possibly utilized by third party plugin in Jira for triggering some other actions to happen.

I suspect that structure is trying to perform some huge amount of work and getting really bogged down the in the process, but it's unclear to as to the details of how or why. 

I hope this helps,

Andy

Hi Knut,

As Andy suggested, please, contact us at our support portal or by email support@almworks.com and we'll be happy to help you. The first thing to check from our side would be Performance Audit Log (found under Administration>Structure>Support), we'll let you know in the ticket what other logs and checks we'll need.

Regards,
Egor Tasa

ALM Works

this was the wrong place for my answer, thats the cause i want to remove this reply, but i can`t :/

Hi there,

the investigation of the structure support team reached out that the structure-plugin doesn’t seem to be the culprit in the performance problem. 

 

Here is the link for the coressponding structure-support-case:

https://support.almworks.com/servicedesk/customer/portal/8/ALM-15022

 

This link is for the last generated threaddumps: https://www.dropbox.com/s/4dc2ig93puvtpfg/jira-threaddumps.zip?dl=0

And here are the corresponding jira support logs: https://www.dropbox.com/s/vh7cbrs9ue4l6yf/Jira_support_2020-02-13-09-40-04.zip?dl=0

 

I`m looking forward for any help.

 

Best regards,

-Knut

Hi @Knut_Luebke 

As mentioned in other post, here's the summary of our latest investigation.

1. Judging by the Performance Audit Log, there are no structures with misconfigurations, and they are all presenting a fast generation time.

2. From the thread dumps, we see a few threads blocking other threads (This might be the reason for the slowness).

org.apache.log4j.RollingFileAppender is blocking 8 threads.

java.lang.Object is blocking 5 threads.

java.util.concurrent.locks.ReentrantLock$NonfairSync is blocking 2 threads.

java.util.WeakHashMap is blocking 1 thread.

oracle.jdbc.driver.T4CConnection is blocking 1 thread.

These threads might be an indication of the following bug:

https://jira.atlassian.com/browse/JRASERVER-62958

3. We were also able to observe some database errors taking place.


2020-03-17 08:47:58,779 http-nio-8090-exec-16 ERROR knl 527x165x1 vcpnc2 10.160.1.55 /secure/AjaxIssueEditAction!default.jspa [c.d.j.p.impl.field.FieldServiceImpl] There was a SQL exception thrown by the Active Objects library:
Database:
- name:Oracle
- version:Oracle Database 12c Standard Edition Release 12.1.0.2.0 - 64bit Production
- minor version:1
- major version:12
Driver:
- name:Oracle JDBC driver
- version:12.1.0.1.0 java.sql.SQLException: Protocol violation: [ 19, 227, ]
com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
- name:Oracle
- version:Oracle Database 12c Standard Edition Release 12.1.0.2.0 - 64bit Production
- minor version:1
- major version:12
Driver:
- name:Oracle JDBC driver
- version:12.1.0.1.0java.sql.SQLException: Protocol violation: [ 19, 227, ]

There is a KB from Confluence mentioning this error:

[java.sql.SQLException: Protocol violation caught while accessing a page and Oracle DB is used|https://confluence.atlassian.com/confkb/java-sql-sqlexception-protocol-violation-caught-while-accessing-a-page-and-oracle-db-is-used-781393258.html]

Even though the KB relates to Confluence, the root cause appears to be the database driver, so it could be the same issue.

Conclusion and next steps:

I'd suggest looking into the mentioned bug and also investigate the database errors.

Judging by all the files, Structure is not playing any role in the experienced issue.

Let me know if you have any questions.

Mario O.G
[ALM Works]

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Jira Software

How to create Jira issus from Excel file?

When to use CSV importer When managing your processes in Jira, there are many occasions where you need to create a lot of tasks. Creating them one by one will cost you a lot of time and effort and i...

4,623 views 22 33
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you