Hello,
I found a very similar topic here, but the solution there didn´t solve our problem.
We have a very old Jira instance running on AWS infrastructure (EC2-Instance with Amazon Linux). We want to update it to the newest version, therefore we want small update steps, to avoid big update problems.
Update from 7.1.4 to 7.2.15 - After running the installer and startup the instance the instance failed to start up with this error message:
Here an excerpt from the "logs/catalina.out" Logfile:
[..]
2018-07-23 12:11:46,571 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
***********************************************************************************
JIRA 7.2.15 build: 72018 started. You can now access JIRA through your web browser.
***********************************************************************************
2018-07-23 12:11:46,588 JIRA-Bootstrap INFO [c.a.plugin.manager.DefaultPluginManager] Plugin system lateStartup begun
2018-07-23 12:11:46,591 JIRA-Bootstrap INFO [c.a.plugin.manager.DefaultPluginManager] Plugin system lateStartup ended
2018-07-23 12:11:46,594 JIRA-Bootstrap DEBUG [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.plugins.atlassian-whitelist-api-plugin]
2018-07-23 12:11:46,625 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
___ Plugin System Started _________________
2018-07-23 12:11:46,625 JIRA-Bootstrap ERROR [c.a.jira.upgrade.UpgradeLauncher] Skipping, JIRA is locked.
2018-07-23 12:11:46,626 JIRA-Bootstrap INFO [c.a.jira.scheduler.JiraSchedulerLauncher] JIRA Scheduler not started: JIRA startup checklist failed.
2018-07-23 12:11:46,678 JIRA-Bootstrap ERROR [c.a.jira.startup.AnalyticsLauncher] Start analytics not scheduled
com.atlassian.scheduler.SchedulerServiceException: Unable to either create or replace clustered job: ImmutableClusteredJob[jobId=03def26b-b3ec-4a6d-afc7-a7b0e102d241,jobRunnerKey=com.atlassian.jira.startup.AnalyticsLauncher.Start,schedule=Schedule[type=INTERVAL,intervalScheduleInfo=IntervalScheduleInfo[firstRunTime=Mon Jul 23 12:14:46 CEST 2018,intervalInMillis=0]],nextRunTime=Mon Jul 23 12:14:46 CEST 2018,version=1,rawParameters=(null)]
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.createOrReplaceWithRetry(CaesiumSchedulerService.java:202)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.scheduleClusteredJob(CaesiumSchedulerService.java:189)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.scheduleJob(CaesiumSchedulerService.java:137)
at com.atlassian.scheduler.core.AbstractSchedulerService.scheduleJobWithGeneratedId(AbstractSchedulerService.java:114)
at com.atlassian.scheduler.core.DelegatingSchedulerService.scheduleJobWithGeneratedId(DelegatingSchedulerService.java:77)
at com.atlassian.jira.startup.AnalyticsLauncher.runStartAnalyticsJob(AnalyticsLauncher.java:72)
at com.atlassian.jira.startup.AnalyticsLauncher.start(AnalyticsLauncher.java:40)
at com.atlassian.jira.startup.ActiveServicesLauncher.start(ActiveServicesLauncher.java:56)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postTenantArrived$4(DefaultJiraLauncher.java:178)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager$StartupTask.run(DefaultInstantUpgradeManager.java:139)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager.runTask(DefaultInstantUpgradeManager.java:55)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager.doNowOrWhenInstanceBecomesActive(DefaultInstantUpgradeManager.java:42)
at com.atlassian.jira.startup.DefaultJiraLauncher.postTenantArrived(DefaultJiraLauncher.java:171)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDBActivated$3(DefaultJiraLauncher.java:159)
at com.atlassian.jira.tenancy.DefaultTenantManager.doNowOrWhenTenantArrives(DefaultTenantManager.java:55)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDBActivated(DefaultJiraLauncher.java:156)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDbLaunch$2(DefaultJiraLauncher.java:145)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:298)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseActivated(DatabaseConfigurationManagerImpl.java:194)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:135)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$start$0(DefaultJiraLauncher.java:101)
at com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:31)
at com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:99)
at com.atlassian.jira.startup.LauncherContextListener.initSlowStuff(LauncherContextListener.java:149)
at java.lang.Thread.run(Thread.java:745)
2018-07-23 12:11:46,745 JIRA-Bootstrap ERROR [c.a.jira.startup.IndexRecoveryLauncher] Index recovery not scheduled
com.atlassian.scheduler.SchedulerServiceException: Unable to either create or replace clustered job: ImmutableClusteredJob[jobId=260b755d-0820-42db-9726-3e1d91c91d50,jobRunnerKey=com.atlassian.jira.DefaultJiraLauncher.IndexRecovery,schedule=Schedule[type=INTERVAL,intervalScheduleInfo=IntervalScheduleInfo[firstRunTime=Mon Jul 23 12:11:46 CEST 2018,intervalInMillis=0]],nextRunTime=Mon Jul 23 12:11:46 CEST 2018,version=1,rawParameters=(null)]
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.createOrReplaceWithRetry(CaesiumSchedulerService.java:202)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.scheduleClusteredJob(CaesiumSchedulerService.java:189)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.scheduleJob(CaesiumSchedulerService.java:137)
at com.atlassian.scheduler.core.AbstractSchedulerService.scheduleJobWithGeneratedId(AbstractSchedulerService.java:114)
at com.atlassian.scheduler.core.DelegatingSchedulerService.scheduleJobWithGeneratedId(DelegatingSchedulerService.java:77)
at com.atlassian.jira.startup.IndexRecoveryLauncher.runIndexRecoveryJob(IndexRecoveryLauncher.java:102)
at com.atlassian.jira.startup.IndexRecoveryLauncher.start(IndexRecoveryLauncher.java:49)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postTenantArrived$4(DefaultJiraLauncher.java:181)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager$StartupTask.run(DefaultInstantUpgradeManager.java:139)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager.runTask(DefaultInstantUpgradeManager.java:55)
at com.atlassian.jira.startup.DefaultInstantUpgradeManager.doNowOrWhenInstanceBecomesActive(DefaultInstantUpgradeManager.java:42)
at com.atlassian.jira.startup.DefaultJiraLauncher.postTenantArrived(DefaultJiraLauncher.java:171)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDBActivated$3(DefaultJiraLauncher.java:159)
at com.atlassian.jira.tenancy.DefaultTenantManager.doNowOrWhenTenantArrives(DefaultTenantManager.java:55)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDBActivated(DefaultJiraLauncher.java:156)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDbLaunch$2(DefaultJiraLauncher.java:145)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:298)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseActivated(DatabaseConfigurationManagerImpl.java:194)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:135)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$start$0(DefaultJiraLauncher.java:101)
at com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:31)
at com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:99)
at com.atlassian.jira.startup.LauncherContextListener.initSlowStuff(LauncherContextListener.java:149)
at java.lang.Thread.run(Thread.java:745)
2018-07-23 12:11:46,755 JIRA-Bootstrap INFO [c.a.jira.startup.DefaultInstantUpgradeManager] Late startup launchers took 0s
2018-07-23 12:11:46,760 JIRA-Bootstrap ERROR [c.a.jira.startup.LauncherContextListener] Unable to start JIRA.
java.lang.IllegalStateException: Abnormal system startup detected
at com.atlassian.jira.startup.LauncherContextListener.initDone(LauncherContextListener.java:180)
at com.atlassian.jira.startup.LauncherContextListener.initSlowStuff(LauncherContextListener.java:162)
at java.lang.Thread.run(Thread.java:745)
2018-07-23 12:11:46,775 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Memory Usage:
---------------------------------------------------------------------------------
Heap memory : Used: 418 MiB. Committed: 1602 MiB. Max: 1946 MiB
Non-heap memory : Used: 224 MiB. Committed: 244 MiB. Max: 1264 MiB
---------------------------------------------------------------------------------
TOTAL : Used: 642 MiB. Committed: 1846 MiB. Max: 3210 MiB
---------------------------------------------------------------------------------
2018-07-23 12:11:48,826 Modification Check:thread-1 INFO [c.a.jira.startup.JiraStartupLogger]
___ Modifications ___________________________
Modified Files : jira-application.properties
Removed Files : None
To solve this I tried the following things (found in this three articles)
- https://community.atlassian.com/t5/Jira-Core-questions/JIRA-Startup-Failed-Unable-to-either-create-or-replace-clustered/qaq-p/686280
- https://community.atlassian.com/t5/Jira-Software-questions/Unable-to-start-JIRA/qaq-p/726633
-https://confluence.atlassian.com/jirakb/troubleshooting-jira-startup-failed-error-394464512.html
I run these to commands on our database:
jiradb=# reindex database "jiradb";
jiradb=# vacuum analyze public.clusteredjob
This did not work, then I delete the "clustereddjob" table complete, also this didn't work. (There were no duplicate rows in this table)
Also, I tried:
- Increasing JIRA Memory.
- Start Jira in Safe Mode (all plugins disabled)
- Plugins require more time to startup (add 30sec wait)
-Datlassian.plugins.enable.wait=300
Can you help me with this problem?
