Not able to access Jira dashboard

Question

Hi,

Since morning not able to access Jira dashboard, I did restart and noticed below details in catalina.out, seems Jira startup checks passed and ready to server. later tomcat thread got stuck

2021-07-21 06:39:17,042+0000 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /jbs/apps/webapps/bugsjira/8.13.6/home/dbconfig.xml
2021-07-21 06:39:17,085+0000 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Trying to get encrypted password from xml and decrypt it
2021-07-21 06:39:17,087+0000 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Database password decryption not performed.
Jul 21, 2021 6:39:17 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-nio-8080"]
Jul 21, 2021 6:39:17 AM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector
INFO: Using a shared selector for servlet write/read
Jul 21, 2021 6:39:17 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 5064 ms
2021-07-21 06:39:17,783+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] Running Jira startup checks.
2021-07-21 06:39:17,784+0000 JIRA-Bootstrap FATAL [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.
2021-07-21 06:39:17,849+0000 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Startup is complete. Jira is ready to serve.
2021-07-21 06:39:17,853+0000 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Memory Usage:
---------------------------------------------------------------------------------
Heap memory : Used: 147 MiB. Committed: 4096 MiB. Max: 16384 MiB
Non-heap memory : Used: 60 MiB. Committed: 74 MiB. Max: 2552 MiB
---------------------------------------------------------------------------------
TOTAL : Used: 206 MiB. Committed: 4170 MiB. Max: 18936 MiB
---------------------------------------------------------------------------------
Jul 21, 2021 6:41:47 AM org.apache.catalina.valves.StuckThreadDetectionValve notifyStuckThreadDetected
WARNING: Thread [http-nio-8080-exec-6] (id=[39]) has been active for [120,622] milliseconds (since [7/21/21 6:39 AM]) to serve the same request for [https://bugs-stage.openjdk.java.net/secure/errors.jsp] and may be stuck (configured threshold for this StuckThreadDetectionValve is [120] seconds). There is/are [1] thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:475)
at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:469)
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:69)

Daniel Ebers · Answer

Hi @Pandiyan Muthuraman

I am wondering if the errors you see are only results from a superior error.

In log already at timestamp 06:39:17 a FATAL error is locked leading to "Jira is locked" state.

2021-07-21 06:39:17,784+0000 JIRA-Bootstrap FATAL [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.

The error messages (they are more prominent, longer and look critical by their own sense) are generated later.

The root cause is not clear - but have you had a look into atlassian-jira,log also?
Hopefully there is more detailed output.

Also, can you during the startup already open a browser and point it to Jira as a normal user would? Sometimes even in an early state of Jira starting useful hints are visible in browser as to why it comes locked.

Regards,
Daniel

Pandiyan Muthuraman · Answer

I did restart again, now seeing "memory leak" error

07-22 17:02:45,123+0000 JIRA-Bootstrap INFO      [c.a.jira.startup.JiraStartupLogger] Running Jira startup checks.
2021-07-22 17:02:45,123+0000 JIRA-Bootstrap FATAL      [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.
2021-07-22 17:02:45,213+0000 JIRA-Bootstrap INFO      [c.a.jira.startup.LauncherContextListener] Startup is complete. Jira is ready to serve.
2021-07-22 17:02:45,215+0000 JIRA-Bootstrap INFO      [c.a.jira.startup.LauncherContextListener] Memory Usage:
    ---------------------------------------------------------------------------------
      Heap memory     :  Used:  835 MiB.  Committed: 4096 MiB.  Max: 16384 MiB
      Non-heap memory :  Used:   74 MiB.  Committed:   83 MiB.  Max: 2552 MiB
    ---------------------------------------------------------------------------------
      TOTAL           :  Used:  909 MiB.  Committed: 4179 MiB.  Max: 18936 MiB
    ---------------------------------------------------------------------------------
2021-07-22 17:02:45,216+0000 localhost-startStop-2 INFO      [c.a.jira.startup.DefaultJiraLauncher] Stopping launchers
Jul 22, 2021 5:02:45 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [ROOT] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
Jul 22, 2021 5:02:45 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [ROOT] appears to have started a thread named [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 oracle.jdbc.driver.BlockSource$ThreadedCachingBlockSource$BlockReleaser.run(BlockSource.java:329)
Jul 22, 2021 5:02:45 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [ROOT] appears to have started a thread named [InterruptTimer] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
Jul 22, 2021 5:02:45 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks

Suhas P · Answer

Are you using a reverse-proxy such as apache or Nginx? It seems like the thread is being killed as it reaches the threshold. Try increasing the threshold in the reverseproxy/load balancer config file and restart the application again.