Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Jira application is down

Hi,

.jira.startup.IndexRecoveryLauncher] Issue index is out of date with the database by more than 24 hours. Automatic recovery is not attempted

 

Facing this error after upgrading my jira from 8.4 to 8.7,please let me know how to resolve this

We have upgraded our DB to postgres 11.10 after the jira upgrade

Kindly help me out on resolving this issue.

 

Thanks

Priya K

3 answers

0 votes

That error will not be stopping Jira from running, it's telling you there are data inconsistencies, and you really need to re-index your Jira in full.

What error messages are you getting that are stopping Jira?

If you have issues with indexing after upgrade of Postgres to 11.10 you can look at https://confluence.atlassian.com/jirakb/reindex-progresses-very-slowly-after-postgres-upgrade-or-restore-1070073091.html

I had to vacuum and reindex the database after the upgrade to make Jira work properly

@Jonas Ekström @Nic Brough _Adaptavist_ @a.gowri82@gmail.com

 

Hi ,

 

We have performed the vaccum full and reindex in database, but still the jira site is down,

kindly see the error am facing out.

JIRA 8.7.0 build: 807000 started. You can now access JIRA through your web browser.
    ***********************************************************************************2021-07-21 05:53:57,206-0400 JIRA-Bootstrap INFO      [c.a.jira.startup.JiraStartupLogger]    ___ Plugin System Started _________________2021-07-21 05:53:57,286-0400 JIRA-Bootstrap INFO      [c.a.jira.upgrade.UpgradeScheduler] Scheduling upgrades to run in 1 minute(s)
2021-07-21 05:53:59,143-0400 Modification Check:thread-1 INFO      [c.a.jira.startup.JiraStartupLogger]    ___ Modifications ___________________________         Modified Files                                : jira-application.properties
         Removed Files                                 : None2021-07-21 05:54:02,799-0400 JIRA-Bootstrap INFO      [c.a.jira.startup.LauncherContextListener] Memory Usage:
    ---------------------------------------------------------------------------------
      Heap memory     :  Used:  757 MiB.  Committed: 1817 MiB.  Max: 4004 MiB
      Non-heap memory :  Used:  276 MiB.  Committed:  302 MiB.  Max: 1536 MiB
    ---------------------------------------------------------------------------------
      TOTAL           :  Used: 1033 MiB.  Committed: 2119 MiB.  Max: 5540 MiB
    ---------------------------------------------------------------------------------
2021-07-21 05:54:02,895-0400 Caesium-1-4 INFO anonymous    My reminders Service [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-21 05:54:02,967-0400 Caesium-1-1 INFO ServiceRunner     [c.a.jira.startup.IndexRecoveryLauncher] Running start-index-recovery
2021-07-21 05:54:03,276-0400 Caesium-1-4 INFO anonymous    My reminders Service [c.a.jira.startup.CacheWarmerLauncher] Warming up 3 caches.
2021-07-21 05:54:03,976-0400 Caesium-1-1 ERROR ServiceRunner     [c.a.jira.startup.IndexRecoveryLauncher] Issue index is out of date with the database by more than 24 hours. Automatic recovery is not attempted
2021-07-21 05:54:13,163-0400 Caesium-1-4 INFO anonymous    My reminders Service [c.a.jira.startup.CacheWarmerLauncher] Warmed cache(s) in 9886 ms.
2021-07-21 05:54:57,291-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] Running scheduled upgrades
2021-07-21 05:54:57,308-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has started
2021-07-21 05:54:57,529-0400 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting current build number to 807000
2021-07-21 05:54:57,532-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeIndexManager] There are no reindex requests of type [IMMEDIATE, DELAYED] so none will be run
2021-07-21 05:54:57,532-0400 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting current version to 8.7.0
2021-07-21 05:54:57,533-0400 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting downgrade version to 7.1.2
2021-07-21 05:54:57,539-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has finished successfully, and took 230 milliseconds to process.
2021-07-21 05:54:57,539-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] JIRA upgrades completed successfully
2021-07-21 05:54:57,545-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] Plugins upgrades completed successfully
2021-07-21 05:54:57,546-0400 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeIndexManager] Reindexing is not allowed after this upgrade and there is no immediate reindex requests

 

to perform the reindex through the application my site is not up.

Kindly suggest the method to overcome this.

 

Thanks

Priya K

0 votes

So, the "You can now access JIRA through your web browser." belies your original "Jira is down" question.  It's not, according to the log, it is running.

The rest of the log is not showing any errors that would halt Jira either, so you'll need to do some debugging.

What exactly do you mean by "Jira is down"?  What happens when you try to browse to the site?  Churning?  Errors?  Warnings?  Corrupted Jira login screen?

On the server side, is it actually running, as the log file implies?  Is the log really just stopping at that point?

@Nic Brough _Adaptavist_

@Nic Brough _Adaptavist_ @Jonas Ekström 

Hi,

This is the screen when i try to access the jira site.Am not sure whats blocking my site.Capture.PNG

Ok, a 408 is a timeout, which implies there is something there, but it's not responding to anything after the initial "yep, give me a sec to get some data for you" that a page sends back to your browser when first contacted (you'd get a 404 if there was no response to the first contact)

The thing that is running there will be the Tomcat (which will be asking the Jira it is running for the actual content to serve up).

So you'll need to look at why Tomcat or Jira are not serving up.

I would start with the application log, where you've looked already, but at the actual errors that are a result of it crashing.  If it really is stopping dead where you've cut it off in your post, then there is a fault happening in the application server, not the application, and you'll want to read the Tomcat log at <jira install>/logs/catalina.out to find out why it's hard-killing Jira.

@Nic Brough _Adaptavist_ 

 

Please find the complete log details,still i couldnt access my site

 

2021-07-21 08:03:37,417-0400 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]

___ System-level feature flags ______________

com.atlassian.jira.agile.darkfeature.editable.detailsview : true
jira.zdu.admin-updates-ui : true
nps.survey.inline.dialog : true
jira.zdu.jmx-monitoring : true
jira.plugin.devstatus.phasetwo : true
jira.frother.reporter.field : true
atlassian.rest.xsrf.legacy.enabled : true
jira.issue.status.lozenge : true
com.atlassian.jira.config.BIG_PIPE : true
sd.new.settings.sidebar.location.disabled : true
jira.zdu.cluster-upgrade-state : true
com.atlassian.jira.config.PDL : true
jira.plugin.devstatus.phasetwo.enabled : true
atlassian.aui.raphael.disabled : true
app-switcher.new : true
frother.assignee.field : true
jira.onboarding.cyoa : true
com.atlassian.jira.config.CoreFeatures.LICENSE_ROLES_ENABLED : true
com.atlassian.jira.config.ProjectConfig.MENU : true
com.atlassian.jira.projects.sidebar.DEFER_RESOURCES : true
jira.export.csv.enabled : true

2021-07-21 08:03:37,418-0400 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]

***********************************************************************************
JIRA 8.7.0 build: 807000 started. You can now access JIRA through your web browser.
***********************************************************************************

2021-07-21 08:03:38,069-0400 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]

___ Plugin System Started _________________

2021-07-21 08:03:38,166-0400 JIRA-Bootstrap INFO [c.a.jira.upgrade.UpgradeScheduler] Scheduling upgrades to run in 1 minute(s)
2021-07-21 08:03:40,627-0400 Modification Check:thread-1 INFO [c.a.jira.startup.JiraStartupLogger]

___ Modifications ___________________________

Modified Files : jira-application.properties
Removed Files : None

2021-07-21 08:03:43,246-0400 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Memory Usage:
---------------------------------------------------------------------------------
Heap memory : Used: 610 MiB. Committed: 1781 MiB. Max: 4020 MiB
Non-heap memory : Used: 279 MiB. Committed: 304 MiB. Max: 1536 MiB
---------------------------------------------------------------------------------
TOTAL : Used: 889 MiB. Committed: 2084 MiB. Max: 5556 MiB
---------------------------------------------------------------------------------
2021-07-21 08:03:43,305-0400 Caesium-1-1 INFO [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-21 08:03:43,664-0400 Caesium-1-3 INFO [c.a.jira.startup.IndexRecoveryLauncher] Running start-index-recovery
2021-07-21 08:03:43,700-0400 Caesium-1-1 INFO [c.a.jira.startup.CacheWarmerLauncher] Warming up 3 caches.
2021-07-21 08:03:44,674-0400 Caesium-1-3 ERROR [c.a.jira.startup.IndexRecoveryLauncher] Issue index is out of date with the database by more than 24 hours. Automatic recovery is not attempted
2021-07-21 08:03:53,375-0400 Caesium-1-1 INFO [c.a.jira.startup.CacheWarmerLauncher] Warmed cache(s) in 9674 ms.
2021-07-21 08:04:38,169-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.UpgradeScheduler] Running scheduled upgrades
2021-07-21 08:04:38,194-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has started
2021-07-21 08:04:38,416-0400 Caesium-1-1 INFO ServiceRunner [c.a.j.upgrade.util.BuildNumberDao] Setting current build number to 807000
2021-07-21 08:04:38,419-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.UpgradeIndexManager] There are no reindex requests of type [IMMEDIATE, DELAYED] so none will be run
2021-07-21 08:04:38,419-0400 Caesium-1-1 INFO ServiceRunner [c.a.j.upgrade.util.BuildNumberDao] Setting current version to 8.7.0
2021-07-21 08:04:38,419-0400 Caesium-1-1 INFO ServiceRunner [c.a.j.upgrade.util.BuildNumberDao] Setting downgrade version to 7.1.2
2021-07-21 08:04:38,423-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has finished successfully, and took 230 milliseconds to process.
2021-07-21 08:04:38,423-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.UpgradeScheduler] JIRA upgrades completed successfully
2021-07-21 08:04:38,430-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.UpgradeScheduler] Plugins upgrades completed successfully
2021-07-21 08:04:38,431-0400 Caesium-1-1 INFO ServiceRunner [c.a.jira.upgrade.UpgradeIndexManager] Reindexing is not allowed after this upgrade and there is no immediate reindex requests
2021-07-21 20:03:43,304-0400 Caesium-1-3 INFO anonymous My reminders Service [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-22 08:03:43,304-0400 Caesium-1-3 INFO ServiceRunner [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-22 20:03:43,311-0400 Caesium-1-1 INFO ServiceRunner [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-23 08:03:43,303-0400 Caesium-1-2 INFO anonymous My reminders Service [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-23 20:03:43,305-0400 Caesium-1-2 INFO ServiceRunner [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2021-07-24 08:03:43,303-0400 Caesium-1-1 INFO anonymous My reminders Service [c.a.r.internal.configuration.ConfigurationLoggerJob] Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
[jira@jira logs]$

 

Thanks

Priya 

Suggest an answer

Log in or Sign up to answer
TAGS

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you