Not able to restore system or perform reindex after failed restore

[RCSLT Admin]

I deleted a project accidentally and when I went to restore via JIRA XML I got a 500 error and after unable to reindex.

Not sure how I can fix the system as loads of it is broken. 

I cannot restore or reindex. Please help - I still am entitled to support but Atlassian are making it impossible to raise a ticket!

Error when trying to re-index:

java.lang.NullPointerException

at com.atlassian.jira.web.filters.JiraImportProgressFilter.doFilter(JiraImportProgressFilter.java:117) [classes/:?]

at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32) [atlassian-core-5.0.6.jar:?]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) [catalina.jar:8.5.6]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [catalina.jar:8.5.6]

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [catalina.jar:8.5.6]

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) [catalina.jar:8.5.6]

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [catalina.jar:8.5.6]

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.5.6]

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [catalina.jar:8.5.6]

at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) [catalina.jar:8.5.6]

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) [catalina.jar:8.5.6]

at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:784) [tomcat-coyote.jar:8.5.6]

at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-coyote.jar:8.5.6]

at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:802) [tomcat-coyote.jar:8.5.6]

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410) [tomcat-coyote.jar:8.5.6]

at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:8.5.6]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.5.6]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

[Andy Heinzer]

Hi Navid,

In regards to opening a support case, back in last July, Atlassian changed the support offerings for starter licenses.   Since that time, support for these licenses are provided directly here in Community.   More details on this change are explained in the blog We’re updating our support offering for server starter (10-user) licenses .

Sorry to hear there is a problem with trying to restore this data.  I am afraid that this stacktrace does not really indicate a clear cause of the problem here.

For the next steps, I would actually recommend creating a new blank database per our instructions on Connecting JIRA applications to a database.   And then connecting your Jira instance to this new empty database.  When you then start Jira, it will launch the initial setup wizard.  This wizard has the option to 'import your data' link where you can then specify the xml backup file.   This is another way to restore Jira data.   I would be interested to learn if perhaps this method can both restore your data and get your Jira instance back to a working state.

If this does not work, please let me know.  If need be we can generate a separate support case to gather a complete set of logs to try to investigate this further.  The advantage of this method is that we would have a separate database to try these changes against.   If this does not work, we can still reconnect Jira to your previous database and attempt to troubleshoot the initial problem as well.

Regards,

Andy

[RCSLT Admin]

Hi Andy 

Thanks for the reply.  We will try that and get back to you.  

I checked the health check results and found the following:

Name: Embedded database
Is healthy: true
Failure reason: You are not using an HSQL or H2 embedded database with a production license.
Severity: UNDEFINED

Name: End of Life
Is healthy: true
Failure reason: JIRA version 7.3.x has not reached End of Life. This version will reach End of Life in 273 days.
Severity: UNDEFINED

Name: Base URL for gadgets
Is healthy: true
Failure reason: JIRA is able to access itself through the configured Base URL.
Severity: UNDEFINED

Name: Supported database
Is healthy: true
Failure reason: Your database model and version are supported.
Severity: UNDEFINED

Name: Collation
Is healthy: true
Failure reason: The database collation 'English_United Kingdom.1252' is unsupported by JIRA. The Table collation 'DEFAULT' is supported by JIRA. A limited number of circumstances could cause unexpected results.
Severity: UNDEFINED

Name: Connection settings
Is healthy: true
Failure reason: The database configuration is using our recommended settings.
Severity: UNDEFINED

Name: Lucene
Is healthy: false
Failure reason: The issue index is inconsistent with the database state. The last issue update recorded in the database was at (03/04/18 17:11) but the last issue update recorded in the index was at (04/04/18 13:30).
Severity: MAJOR

Name: Free space
Is healthy: true
Failure reason: The check of disk space for index backup and recovery is ok. Index size is 14 MB and there is at least twice as much free space available.
Severity: UNDEFINED

Name: Attachments
Is healthy: true
Failure reason: The latest attachment (Deputy Chair Role Specification.pdf) for issue (BAU-1331) was found and can be used.
Severity: UNDEFINED

Name: Cluster Index Replication
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Cluster Locks
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Cluster Scheduler
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Cluster Cache Replication
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Cluster Time Synchronisation
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Shared Home
Is healthy: true
Failure reason: The JIRA instance is not clustered.
Severity: UNDEFINED

Name: Attachments
Is healthy: true
Failure reason: Secondary storage is not configured for attachments.
Severity: UNDEFINED

Name: Avatars
Is healthy: true
Failure reason: Secondary storage is not configured for avatars.
Severity: UNDEFINED

Name: Plugins
Is healthy: true
Failure reason: Secondary storage is not configured for plugins.
Severity: UNDEFINED

Name: Index Snapshots
Is healthy: true
Failure reason: Index recovery is not enabled, no index snapshots will be taken.
Severity: UNDEFINED

Name: Lucene index files location
Is healthy: true
Failure reason: Your lucene search index directory 'C:\Program Files\Atlassian\Application Data\JIRA\caches\indexes' is mounted on 'NTFS'.
Severity: UNDEFINED

[Andy Heinzer]

Hi Navid,

From looking at your other post on https://community.atlassian.com/t5/Jira-questions/Problems-after-failed-restore-XML/qaq-p/765498

I think that the problem here is actually that Jira's indexes are corrupted and the application cannot directly correct this right now.

There is a KB that will help with this over in https://confluence.atlassian.com/jirakb/unable-to-rebuild-jira-application-indexes-228623566.html

This will require you to stop Jira and move these files out of this location.  When you then start Jira without these indexes, it should be able to then recreate the indexes based on the data currently in the database.

Try this first, or you can also try this at the same time as restoring to a new database.  The key here is to perform those steps when Jira is stopped.

Andy

[RCSLT Admin]

Hi Andy

Thanks for your advice.  I'm going to speak with our IT support who manage the server where it's hosted and see if they can perform the above steps and get back to you.

Nav

[RCSLT Admin]

Hi Andy

We managed to restore and get the issues/projects back with LUCENE working again but the problem is that none of our Agile/Kanban boards are showing.

Do we need to recreate them or is there a fix?

Thanks for your help!

[RCSLT Admin]

@Andy Heinzer

Some issues face dafter restore:

- Agile board missing

- Unable to render some elements

(all in this issue: https://community.atlassian.com/t5/Jira-Software-questions/JIRA-boards-and-attachments-missing-after-JIRA-System-Restore/qaq-p/767620#U767670)

[Andy Heinzer]

Hi Navid,

Glad to hear the indexing problem has been resolved, but I cannot explain why you might not be seeing your boards at this time.   To further troubleshoot this, I think we need to see the logs from your Jira instance.  Since our community site is not very good at posting files yet, I created this support request on your behalf:

https://getsupport.atlassian.com/servicedesk/customer/portal/20/GHS-113447

I posted some steps you can take there to generate or copy some logs over to us.

Regards,

Andy