We get this message: Memory Utilization is 70.572%, crossed warning (65) or critical (70) threshold.

Cesar Rouillon February 19, 2016
 

17 answers

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 14, 2016

I don't have time to work through all of that for you.  What does it say is happening around the points of high memory load?

 

0 votes
Cesar Rouillon March 14, 2016

I add the appplication logs:

, , , , .

Please your help.

Thanks.

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 11, 2016

That's the application server (Tomcat) log.  No faults in there, so it is healthy.

What does the application log say?

0 votes
Cesar Rouillon March 11, 2016

I add the logs (), please your help.

 

The last erros was in:

Lunes, 15 de febrero de 2016 04:50 p.m.

Miércoles 17 de febrero de 2016 05:45 p.m  

Martes, 23 de febrero de 2016 04:20 p.m.

Miércoles, 02 de marzo de 2016 04:00 p.m.

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 7, 2016

It's in <JIRA home>/logs - the main one is atlassian-jira.log, but there may be others in there that are useful.  It might also be worth looking at the application server (Tomcat) log - that's in <JIRA install>log and the main one is catalina.out

0 votes
Cesar Rouillon March 7, 2016

Where can I find the application log?, what is the directory?

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 4, 2016

Nothing particularly untoward in there, although it does look far too short to be the full thread trace.  It doesn't really point to anything, but it was worth a try.  A thread dump analyser might help read the full one though.

I'm afraid you need to do the analysis on the front end - look at what your users are doing, read the application log to see what is in there at times when the memory usage is high, and monitor the server at those times too (is the disk thrashing?  The database?  The network? etc)

0 votes
Cesar Rouillon March 3, 2016

Dears,

Please your help, until now we get memory utilization critical in the server of jira, we used the command "thread trace", the result is the next:

 

Deadlock Detection:

No deadlocks found.

Thread 3109: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run() @bci=10, line=565 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3108: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- org.apache.jk.common.ChannelSocket.accept(org.apache.jk.core.MsgContext) @bci=46, line=311 (Interpreted frame)
- org.apache.jk.common.ChannelSocket.acceptConnections() @bci=72, line=668 (Interpreted frame)
- org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(java.lang.Object[]) @bci=4, line=879 (Interpreted frame)
- org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() @bci=167, line=690 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3107: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Interpreted frame)
- org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() @bci=26, line=662 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3106: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Interpreted frame)
- org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() @bci=26, line=662 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3105: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Interpreted frame)
- org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() @bci=26, line=662 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3101: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- sun.security.ssl.SSLServerSocketImpl.accept() @bci=42, line=317 (Interpreted frame)
- org.apache.tomcat.util.net.jsse.JSSESocketFactory.acceptSocket(java.net.ServerSocket) @bci=3, line=178 (Interpreted frame)
- org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run() @bci=47, line=352 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3100: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(java.net.ServerSocket) @bci=1, line=61 (Interpreted frame)
- org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run() @bci=47, line=352 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3099: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
- org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() @bci=22, line=1631 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 3017: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Interpreted frame)
- java.util.TimerThread.mainLoop() @bci=28, line=526 (Interpreted frame)
- java.util.TimerThread.run() @bci=1, line=505 (Interpreted frame)


Thread 2074: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Compiled frame)
- org.logicalcobwebs.proxool.PrototyperThread.doWait() @bci=1, line=69 (Compiled frame)
- org.logicalcobwebs.proxool.PrototyperThread.run() @bci=140, line=58 (Compiled frame)


Thread 2073: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
- org.logicalcobwebs.proxool.HouseKeeperThread.run() @bci=71, line=46 (Interpreted frame)


Thread 2072: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
- com.icesoft.util.MonitorRunner$1.run() @bci=14, line=21 (Interpreted frame)


Thread 2065: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
- com.icesoft.faces.webapp.http.servlet.SessionDispatcher$Listener$1.run() @bci=62, line=221 (Interpreted frame)


Thread 2036: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- org.apache.axis.monitor.SOAPMonitorService$ServerSocketThread.run() @bci=9, line=189 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 1807: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept() @bci=1, line=52 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop() @bci=55, line=388 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() @bci=1, line=360 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 1800: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop() @bci=55, line=388 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() @bci=1, line=360 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 1797: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop() @bci=55, line=388 (Interpreted frame)
- sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run() @bci=1, line=360 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 1727: (state = BLOCKED)


Thread 1711: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=135 (Compiled frame)
- java.lang.ref.ReferenceQueue.remove() @bci=2, line=151 (Compiled frame)
- java.lang.ref.Finalizer$FinalizerThread.run() @bci=16, line=189 (Compiled frame)


Thread 1710: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=503 (Compiled frame)
- java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=133 (Compiled frame)


Thread 1688: (state = IN_NATIVE)
- java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
- java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=398 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=530 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=498 (Interpreted frame)
- org.apache.catalina.core.StandardServer.await() @bci=164, line=431 (Interpreted frame)
- org.apache.catalina.startup.Catalina.await() @bci=4, line=676 (Interpreted frame)
- org.apache.catalina.startup.Catalina.start() @bci=194, line=628 (Interpreted frame)
- sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) @bci=0 (Interpreted frame)
- sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=87, line=57 (Interpreted frame)
- sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6, line=43 (Interpreted frame)
- java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=57, line=606 (Interpreted frame)
- org.apache.catalina.startup.Bootstrap.start() @bci=37, line=289 (Interpreted frame)
- org.apache.catalina.startup.Bootstrap.main(java.lang.String[]) @bci=131, line=414 (Interpreted frame)

 

Please your help..

 

Greetings.

 

César

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 24, 2016

One thing that may help is a "thread trace" - you wait until the threshold is crossed again, and then go on to the server and run "jstack PID > outfile", where PID is the process ID of the Tomcat running JIRA and outfile is the log name you want.  When that finishes, the outfile will contain a long list of everything JIRA was doing when you ran it.

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 24, 2016

Yes, you need to look at what your users and/or automated interfaces are doing to JIRA to make it need to use that resource.

The numbers you are quoting indicate load.  They tell us nothing about how your usage is causing that load.

0 votes
Cesar Rouillon February 24, 2016

We don't know why the consume of JIRA in the application server is to much (see the image "Consumo_Recursos_Servidor_Jira.jpg"), beacuse in the database the consume is low. 

Can you get us a recommendation.

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 23, 2016

I'm sorry, I don't mean the resource consumption, I mean you need to look at what your users and automated processes are doing that might be putting load on the system.

0 votes
Cesar Rouillon February 23, 2016

In this image you can look the %CPU of JIRA system.

 

 

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 23, 2016

Not really.  The service is using more memory than the threshold you've set in monitoring.  That doesn't tell you what it's doing or why its doing it. 

The most likely explanation is that the system has come under additional load.  You need to investigate the pattern of usage of your JIRA system.

0 votes
Cesar Rouillon February 23, 2016

Our monitor tool's "OracleCloud" on application server sent us this message:

 

Host=sp12app09.edyficar.local

Target type=Host
Target name=sp12app09.edyficar.local
Incident creation time=Feb 18, 2016 2:23:22 PM PET
Last updated time=Feb 18, 2016 2:23:24 PM PET
Message=Memory Utilization is 70.572%, crossed warning (65) or critical (70) threshold.
Severity=Critical
Incident ID=1421
Event count=1
Incident Status=New
Escalated=No
Priority=High
Incident owner=ADM_ADMINISTRADOR
Incident Acknowledged By Owner=No
Categories=Capacity
Rule Name=Rule_OS,RULE_OS
Rule Owner=SYSMAN
Notification Count=1


We don't know what is the reason for this, is possible that a hide proccess is running?

Thanks!.

 

 

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 19, 2016

Where are you seeing this, and why are suspecting a parameter rather than it being caused by load on your system?

 

0 votes
Cesar Rouillon February 19, 2016

What can we do?, is possible that we can configured some parameter in the JIRA or database?

Thanks

 

 

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events