jira shutdowns after some time

[Nurlan_Rakhimzhanov]

My jira shutdowns approximately after 1 day working.

I did not found any errors in log file.

● jira.service - JIRA Service
 Loaded: loaded (/usr/lib/systemd/system/jira.service; disabled; vendor preset: disabled)
 Active: active (running) since Wed 2017-09-20 13:38:34 +06; 1h 40min ago
 Process: 26844 ExecStop=/path/stop-jira.sh (code=exited, status=1/FAILURE)
 Process: 28139 ExecStart=/path/start-jira.sh (code=exited, status=0/SUCCESS)
 Main PID: 28172 (java)
 CGroup: /system.slice/jira.service
 └─28172 /usr/java/default/jre/bin/java -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m ...

Sep 20 13:38:33 test.server.my start-jira.sh[28139]: .8MMMMM: :NMMMMN.
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: .MMMMMM. .MMMMM~.
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: .MMMMMN .MMMMM?.
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: Atlassian JIRA
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: Version : 7.5.0
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: If you encounter issues starting or stopping JIRA, please see the Troubleshooting guide at http://confluence.atlassian.com/display/JI...ooting+Guide
Sep 20 13:38:33 test.server.my start-jira.sh[28139]: Server startup logs are located in /path/logs/catalina.out
Sep 20 13:38:34 test.server.my start-jira.sh[28139]: Existing PID file found during start.
Sep 20 13:38:34 test.server.my start-jira.sh[28139]: Removing/clearing stale PID file.
Sep 20 13:38:34 test.server.my systemd[1]: Started JIRA Service.
Hint: Some lines were ellipsized, use -l to show in full.

[Nic Brough -Adaptavist-]

What does the catalina.out log say at the time everything stops?

[Nurlan_Rakhimzhanov]

---------------------------------------------------------------------------------
 Heap memory : Used: 421 MiB. Committed: 673 MiB. Max: 746 MiB
 Non-heap memory : Used: 223 MiB. Committed: 243 MiB. Max: 1264 MiB
 ---------------------------------------------------------------------------------
 TOTAL : Used: 644 MiB. Committed: 916 MiB. Max: 2010 MiB
 ---------------------------------------------------------------------------------
2017-09-20 13:40:57,333 Caesium-1-2 INFO [c.a.jira.startup.CacheWarmerLauncher] Warming up 3 caches.
2017-09-20 13:40:57,394 Caesium-1-2 INFO [c.a.jira.i18n.I18nWarmer] Init i18n cache
2017-09-20 13:40:57,423 ForkJoinPool.commonPool-worker-1 INFO [c.a.j.cache.soy.SoyCacheWarmer] Warming 41 soy modules
2017-09-20 13:40:58,459 UpmScheduler:thread-1 WARN anonymous [c.a.upm.pac.PacClientImpl] Update check request may take longer because of the number of add-ons
2017-09-20 13:40:59,078 Caesium-1-3 INFO [c.a.j.index.ha.DefaultIndexRecoveryManager] Latest index date: {2017-09-19 18:15:52}, Latest DB date: {2017-09-19 18:15:52}
2017-09-20 13:41:12,703 Caesium-1-1 DEBUG ServiceRunner [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.plugins.base-hipchat-integration-plugin-api]
2017-09-20 13:41:13,070 Caesium-1-3 INFO [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2017-09-20 13:41:13,071 Caesium-1-3 INFO [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2017-09-20 13:41:22,862 Caesium-1-3 INFO [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...
2017-09-20 13:41:29,176 ForkJoinPool.commonPool-worker-1 INFO [c.a.j.cache.soy.SoyCacheWarmer] Warmed 41 soy module(s) in 31752 ms, 0 module(s) were not present or disabled, 0 failed.
2017-09-20 13:41:30,329 Caesium-1-2 INFO [c.a.jira.i18n.I18nWarmer] Initialised i18n cache in 32934
2017-09-20 13:41:30,329 Caesium-1-2 INFO [c.a.jira.startup.CacheWarmerLauncher] Warmed cache(s) in 32996 ms.
2017-09-20 13:41:30,340 Modification Check:thread-1 INFO [c.a.jira.startup.JiraStartupLogger]

 ___ Modifications ___________________________

 Modified Files : jira-application.properties
 Removed Files : None

2017-09-20 13:41:46,450 Caesium-1-2 INFO [c.a.jira.upgrade.UpgradeScheduler] Running scheduled upgrades
2017-09-20 13:41:46,478 Caesium-1-2 INFO [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has started
2017-09-20 13:41:47,216 Caesium-1-2 INFO [c.a.j.upgrade.util.BuildNumberDao] Setting current build number to 75005
2017-09-20 13:41:47,284 Caesium-1-2 DEBUG [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.jira.plugins.webhooks.jira-webhooks-plugin]
2017-09-20 13:41:47,308 Caesium-1-2 INFO [c.a.jira.upgrade.UpgradeIndexManager] There is no reindex requests of type [IMMEDIATE, DELAYED] so none will be run
2017-09-20 13:41:47,308 Caesium-1-2 INFO [c.a.j.upgrade.util.BuildNumberDao] Setting current version to 7.5.0
2017-09-20 13:41:47,345 Caesium-1-2 INFO [c.a.j.upgrade.util.BuildNumberDao] Setting downgrade version to 7.1.2
2017-09-20 13:41:47,411 Caesium-1-2 INFO [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has finished successfully, and took 932 milliseconds to process.
2017-09-20 13:41:47,412 Caesium-1-2 INFO [c.a.jira.upgrade.UpgradeScheduler] JIRA upgrades completed successfully
2017-09-20 13:41:47,444 Caesium-1-2 INFO [c.a.jira.upgrade.UpgradeScheduler] Plugins upgrades completed successfully
2017-09-20 13:41:47,460 Caesium-1-2 INFO [c.a.jira.upgrade.UpgradeIndexManager] Reindexing is not allowed after this upgrade and there is no immediate reindex requests
2017-09-20 13:41:50,011 http-nio-8180-exec-1 DEBUG - [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.whisper.atlassian-whisper-plugin]
2017-09-20 13:41:53,950 Caesium-1-1 DEBUG ServiceRunner [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.troubleshooting.plugin-jira]
2017-09-20 13:42:30,911 StreamsCompletionService::thread-1 DEBUG karim 822x103x3 1tt72v3 85.29.128.175,127.0.0.1 /plugins/servlet/streams [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.streams.thirdparty-plugin]
20-Sep-2017 13:42:33.471 WARNING [http-nio-8180-exec-25] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/activity-stream/1.0/preferences?_=1505893414742, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
log4j:WARN No appenders could be found for logger (com.atlassian.pocketknife.spi.querydsl.DefaultDialectConfiguration).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
20-Sep-2017 14:12:59.974 WARNING [http-nio-8180-exec-15] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:12:59.973 WARNING [http-nio-8180-exec-4] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/latest/preferredSearchLayout, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:03.939 WARNING [http-nio-8180-exec-18] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:04.137 WARNING [http-nio-8180-exec-3] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:11.490 WARNING [http-nio-8180-exec-15] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:11.644 WARNING [http-nio-8180-exec-22] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:12.776 WARNING [http-nio-8180-exec-21] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:12.958 WARNING [http-nio-8180-exec-10] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:16.063 WARNING [http-nio-8180-exec-23] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:16.263 WARNING [http-nio-8180-exec-8] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:30.511 WARNING [http-nio-8180-exec-13] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/latest/preferredSearchLayout, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:30.511 WARNING [http-nio-8180-exec-14] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable/stable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:33.907 WARNING [http-nio-8180-exec-16] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueNav/operations/views, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:37.610 WARNING [http-nio-8180-exec-2] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueNav/operations/tools, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:13:56.127 WARNING [http-nio-8180-exec-5] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:14:00.723 WARNING [http-nio-8180-exec-8] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 14:14:08.034 WARNING [http-nio-8180-exec-21] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/issueNav/1/issueTable, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
2017-09-20 14:40:09,601 Caesium-1-4 DEBUG ServiceRunner [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.plugins.authentication.atlassian-authentication-plugin]
2017-09-20 14:41:13,050 Caesium-1-3 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2017-09-20 14:41:13,050 Caesium-1-3 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2017-09-20 14:41:22,837 Caesium-1-3 INFO ServiceRunner [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...
20-Sep-2017 15:07:14.575 WARNING [http-nio-8180-exec-13] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/activity-stream/1.0/preferences?_=1505898434493, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 15:25:07.946 WARNING [http-nio-8180-exec-12] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/activity-stream/1.0/preferences?_=1505899507878, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
2017-09-20 15:41:13,049 Caesium-1-2 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2017-09-20 15:41:13,051 Caesium-1-2 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2017-09-20 15:41:22,837 Caesium-1-3 INFO ServiceRunner [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...
2017-09-20 16:41:13,049 Caesium-1-2 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2017-09-20 16:41:13,050 Caesium-1-2 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2017-09-20 16:41:22,837 Caesium-1-3 INFO ServiceRunner [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...
20-Sep-2017 17:36:36.834 WARNING [http-nio-8180-exec-16] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/gadget/1.0/login, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 17:36:39.428 WARNING [http-nio-8180-exec-5] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/activity-stream/1.0/preferences?_=1505907464174, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
20-Sep-2017 17:37:15.535 WARNING [http-nio-8180-exec-4] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://@my-custom-domain@/rest/activity-stream/1.0/preferences?_=1505907500281, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
2017-09-20 17:41:13,049 Caesium-1-4 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2017-09-20 17:41:13,049 Caesium-1-4 INFO ServiceRunner [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2017-09-20 17:41:22,837 Caesium-1-1 INFO ServiceRunner [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...

[Nic Brough -Adaptavist-]

Ok, that's a start up running ok with warnings, but no shut-down information.

That suggests either some serious internal fault, or something in the OS is killing it very hard, not giving it a chance to shut down cleanly.  It's hard to diagnose what that might be, as there's no logs.  Do the application logs say anything, or do they just stop dead when it dies?  Are there any system events in the windows log that might help tell us?

[Nurlan_Rakhimzhanov]

Nothing interesting, seems like system kills jira

[Nic Brough -Adaptavist-]

If it shows no shutdown or critical errors, then yes, it's something in the system killing it.  I'm afraid you'll have to dig through system logs to find out what is doing it.