Jira reindex Timed out

Hi,

I am trying to setup a second staging server for JIRA based on my current setup. I cloned the configurations, files and made all the connections to start JIRA, and it worked out fine. Now when I am trying to reindex, the prgoress stays at 0% for a long time and the log includes a repetitive stack trace. So I had to stop the reindexing, force jira to stop and restart again. I do not see any problem, so I go ahead adn do one more reindexing only to realize that the indexing is locked.


Here is the stack trace form the log:

2017-05-11 13:55:07,654 IssueIndexer:thread-2 DEBUG admin-sboddu 671x177x1 gb43he 10.20.20.253 /secure/admin/jira/IndexReIndex.jspa [c.o.scriptrunner.customfield.GroovyCustomField] com.atlassian.jira.issue.index.SearchUnavailableException
2017-05-11 13:55:07,655 IssueIndexer:thread-9 ERROR admin-sboddu 671x177x1 gb43he 10.20.20.253 /secure/admin/jira/IndexReIndex.jspa [c.a.j.issue.index.DefaultIndexManager] Wait attempt timed out - waited 30000 milliseconds
com.atlassian.jira.issue.index.IndexException: Wait attempt timed out - waited 30000 milliseconds
        at com.atlassian.jira.issue.index.DefaultIndexManager.obtain(DefaultIndexManager.java:794)
        at com.atlassian.jira.issue.index.DefaultIndexManager.access$600(DefaultIndexManager.java:88)
        at com.atlassian.jira.issue.index.DefaultIndexManager$IndexLock.tryLock(DefaultIndexManager.java:1118)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getIndexLock(DefaultIndexManager.java:780)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getEntitySearcher(DefaultIndexManager.java:847)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getIssueSearcher(DefaultIndexManager.java:828)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
        at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:25)
        at com.sun.proxy.$Proxy13.getIssueSearcher(Unknown Source)
        at com.atlassian.jira.issue.search.SearchProviderFactoryImpl.getSearcher(SearchProviderFactoryImpl.java:18)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:278)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:91)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:87)
        at com.atlassian.jira.bc.issue.search.DefaultSearchService.search(DefaultSearchService.java:111)
        at com.atlassian.jira.bc.issue.search.SearchService$search$2.call(Unknown Source)
        at Script4.runJQLSearch(Script4.groovy:106)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
        at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:151)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:336)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:84)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)
        at Script4.retrieveIssuesInEpic(Script4.groovy:70)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
        at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:151)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:336)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:84)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
        at Script4.run(Script4.groovy:38)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:352)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:153)
        at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
        at javax.script.ScriptEngine$eval.call(Unknown Source)
        at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runScriptAndGetContext(ScriptRunnerImpl.groovy:156)
        at com.onresolve.scriptrunner.runner.ScriptRunner$runScriptAndGetContext$5.callCurrent(Unknown Source)
        at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runStringAsScript(ScriptRunnerImpl.groovy:144)
        at com.onresolve.scriptrunner.runner.ScriptRunner$runStringAsScript$4.call(Unknown Source)
        at com.onresolve.scriptrunner.customfield.GroovyCustomField.getValueFromIssue(GroovyCustomField.groovy:248)
        at com.atlassian.jira.issue.fields.CustomFieldImpl.getValue(CustomFieldImpl.java:386)
        at com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFields(SortableTextCustomFieldIndexer.java:39)
        at com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFieldsNotSearchable(SortableTextCustomFieldIndexer.java:34)
        at com.atlassian.jira.issue.index.indexers.impl.AbstractCustomFieldIndexer.addIndex(AbstractCustomFieldIndexer.java:42)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:88)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAll(DefaultIssueDocumentFactory.java:79)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:54)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:34)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:553)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$IndexIssuesOperation.perform(DefaultIssueIndexer.java:377)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$IndexIssuesOperation.perform(DefaultIssueIndexer.java:366)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$null$2(DefaultIssueIndexer.java:305)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$$Lambda$354/1964386317.get(Unknown Source)
        at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:7)
        at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:5)
        at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:33)
        at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:31)
        at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2017-05-11 13:55:07,656 IssueIndexer:thread-9 DEBUG admin-sboddu 671x177x1 gb43he 10.20.20.253 /secure/admin/jira/IndexReIndex.jspa [c.o.scriptrunner.customfield.GroovyCustomField] com.atlassian.jira.issue.index.SearchUnavailableException
2017-05-11 13:55:07,664 IssueIndexer:thread-8 ERROR admin-sboddu 671x177x1 gb43he 10.20.20.253 /secure/admin/jira/IndexReIndex.jspa [c.a.j.issue.index.DefaultIndexManager] Wait attempt timed out - waited 30000 milliseconds
com.atlassian.jira.issue.index.IndexException: Wait attempt timed out - waited 30000 milliseconds
        at com.atlassian.jira.issue.index.DefaultIndexManager.obtain(DefaultIndexManager.java:794)
        at com.atlassian.jira.issue.index.DefaultIndexManager.access$600(DefaultIndexManager.java:88)
        at com.atlassian.jira.issue.index.DefaultIndexManager$IndexLock.tryLock(DefaultIndexManager.java:1118)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getIndexLock(DefaultIndexManager.java:780)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getEntitySearcher(DefaultIndexManager.java:847)
        at com.atlassian.jira.issue.index.DefaultIndexManager.getIssueSearcher(DefaultIndexManager.java:828)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
        at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:25)
        at com.sun.proxy.$Proxy13.getIssueSearcher(Unknown Source)
        at com.atlassian.jira.issue.search.SearchProviderFactoryImpl.getSearcher(SearchProviderFactoryImpl.java:18)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:278)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:91)
        at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:87)
        at com.atlassian.jira.bc.issue.search.DefaultSearchService.search(DefaultSearchService.java:111)
        at com.atlassian.jira.bc.issue.search.SearchService$search$2.call(Unknown Source)
        at Script4.runJQLSearch(Script4.groovy:106)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
        at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:151)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:336)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:84)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)
        at Script4.retrieveIssuesInEpic(Script4.groovy:70)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
        at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:151)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl$2.invokeMethod(GroovyScriptEngineImpl.java:336)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:84)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
        at Script4.run(Script4.groovy:38)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:352)
        at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:153)
        at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
        at javax.script.ScriptEngine$eval.call(Unknown Source)
        at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runScriptAndGetContext(ScriptRunnerImpl.groovy:156)
        at com.onresolve.scriptrunner.runner.ScriptRunner$runScriptAndGetContext$5.callCurrent(Unknown Source)
        at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runStringAsScript(ScriptRunnerImpl.groovy:144)
        at com.onresolve.scriptrunner.runner.ScriptRunner$runStringAsScript$4.call(Unknown Source)
        at com.onresolve.scriptrunner.customfield.GroovyCustomField.getValueFromIssue(GroovyCustomField.groovy:248)
        at com.atlassian.jira.issue.fields.CustomFieldImpl.getValue(CustomFieldImpl.java:386)
        at com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFields(SortableTextCustomFieldIndexer.java:39)
        at com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFieldsNotSearchable(SortableTextCustomFieldIndexer.java:34)
        at com.atlassian.jira.issue.index.indexers.impl.AbstractCustomFieldIndexer.addIndex(AbstractCustomFieldIndexer.java:42)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:88)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAll(DefaultIssueDocumentFactory.java:79)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:54)
        at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:34)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:553)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$IndexIssuesOperation.perform(DefaultIssueIndexer.java:377)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$IndexIssuesOperation.perform(DefaultIssueIndexer.java:366)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$null$2(DefaultIssueIndexer.java:305)
        at com.atlassian.jira.issue.index.DefaultIssueIndexer$$Lambda$354/1964386317.get(Unknown Source)
        at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:7)
        at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:5)
        at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:33)
        at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:31)
        at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

I believe this error is caused due to either one of the two problems:

1. As I tried to stop the initial reindex, the whole jira instance became non-responsive and it is unable to get out from the locked state ( even after I restart jira). 

2. I strongly believe that one of my custom field has failed and it is causing this error. The stack trace points to a Scripted Field (highlighted fields in the above log) that is custom to our workflow setup.

 

Please let me know how to diagnose this issue.

Thank You!

1 answer

Hello Invinci:

This looks like a known error from the Lucene index. You can find more information about how to fix it here

Could you give us your Jira version? We would like to attempt to reproduce this.

Cheers!

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted Oct 11, 2018 in Marketplace Apps

You + one app + a desert island...

Hi all! My name is Miles and I work on the Marketplace team. We’re looking for better ways to recommend and suggest apps that are truly crowd favorites, so of course we wanted to poll the Community. ...

3,681 views 6 6
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you