Deadlock with HashMap and ScripRunner

Mike Wells
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 18, 2015

I updated from JIRA 6.3.7 to 6.4.6, when I start JIRA I get the following exception...

2015-06-18 11:43:17,183 DeadlockDetection:thread-1 ERROR      [atlassian.jira.startup.LauncherContextListener] A deadlock has been detected on JIRA startup for the following threads: ["Thread-34" Id=100 BLOCKED on java.util.HashMap@4c9460af owned by "localhost-startStop-1" Id=13, "localhost-startStop-1" Id=13 BLOCKED on groovy.util.GroovyScriptEngine$ScriptClassLoader@cedc0af owned by "Thread-34" Id=100]
2015-06-18 11:43:17,185 DeadlockDetection:thread-1 ERROR      [atlassian.jira.startup.LauncherContextListener] "Thread-34" Id=100 BLOCKED on java.util.HashMap@4c9460af owned by "localhost-startStop-1" Id=13
    at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:735)
    -  blocked on java.util.HashMap@4c9460af
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:696)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:765)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:753)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at java_lang_Class$forName$1.call(Unknown Source)
    at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.loadScriptByName(ScriptRunnerImpl.groovy:583)
    ...:
    groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:735)
    groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:696)
    groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:765)
    groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:753)
    java.lang.Class.forName0(Native Method)
    java.lang.Class.forName(Unknown Source)
    java_lang_Class$forName$1.call(Unknown Source)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.loadScriptByName(ScriptRunnerImpl.groovy:583)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
    org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:368)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
    org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:145)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl$_listScripts_closure8_closure11.doCall(ScriptRunnerImpl.groovy:650)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
    org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:278)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
    groovy.lang.Closure.call(Closure.java:423)
    groovy.lang.Closure.call(Closure.java:439)
    org.codehaus.groovy.runtime.ResourceGroovyMethods.eachFileRecurse(ResourceGroovyMethods.java:1099)
    org.codehaus.groovy.runtime.ResourceGroovyMethods.eachFileRecurse(ResourceGroovyMethods.java:1331)
    org.codehaus.groovy.runtime.dgm$792.invoke(Unknown Source)
    org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)
    org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl$_listScripts_closure8.doCall(ScriptRunnerImpl.groovy:639)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
    org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:278)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
    groovy.lang.Closure.call(Closure.java:423)
    groovy.lang.Closure.call(Closure.java:439)
    org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1379)
    org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1372)
    org.codehaus.groovy.runtime.dgm$149.invoke(Unknown Source)
    org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:271)
    org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.listScripts(ScriptRunnerImpl.groovy:636)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.listScriptsSuppressErrors(ScriptRunnerImpl.groovy:618)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.listScriptsSuppressErrors(ScriptRunnerImpl.groovy)
    com.onresolve.scriptrunner.runner.ScriptRunner$listScriptsSuppressErrors$0.call(Unknown Source)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    com.onresolve.scriptrunner.runner.JqlFunctionsManagerImpl.registerScriptedJqlFunctions(JqlFunctionsManagerImpl.groovy:198)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)
    org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnCurrentN(ScriptBytecodeAdapter.java:78)
    com.onresolve.scriptrunner.runner.JqlFunctionsManagerImpl.this$dist$invoke$1(JqlFunctionsManagerImpl.groovy)
    com.onresolve.scriptrunner.runner.JqlFunctionsManagerImpl$RegisterJqlFunctions.methodMissing(JqlFunctionsManagerImpl.groovy)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    groovy.lang.MetaClassImpl.invokeMissingMethod(MetaClassImpl.java:933)
    groovy.lang.MetaClassImpl.invokePropertyOrMissing(MetaClassImpl.java:1256)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1209)
    groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
    org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137)
    com.onresolve.scriptrunner.runner.JqlFunctionsManagerImpl$RegisterJqlFunctions.run(JqlFunctionsManagerImpl.groovy:270)
    java.lang.Thread.run(Unknown Source)
2015-06-18 11:43:17,186 DeadlockDetection:thread-1 ERROR      [atlassian.jira.startup.LauncherContextListener] "localhost-startStop-1" Id=13 BLOCKED on groovy.util.GroovyScriptEngine$ScriptClassLoader@cedc0af owned by "Thread-34" Id=100
    at java.lang.ClassLoader.loadClass(Unknown Source)
    -  blocked on groovy.util.GroovyScriptEngine$ScriptClassLoader@cedc0af
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:655)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:523)
    at org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(ClassNodeResolver.java:183)
    at org.codehaus.groovy.control.ClassNodeResolver.findClassNode(ClassNodeResolver.java:168)
    at groovy.util.GroovyScriptEngine$ScriptClassLoader$3.findClassNode(GroovyScriptEngine.java:218)
    at org.codehaus.groovy.control.ClassNodeResolver.resolveName(ClassNodeResolver.java:124)
    at org.codehaus.groovy.control.ResolveVisitor.resolveToOuter(ResolveVisitor.java:617)
    ...:
    java.lang.ClassLoader.loadClass(Unknown Source)
    groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:655)
    groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:523)
    org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(ClassNodeResolver.java:183)
    org.codehaus.groovy.control.ClassNodeResolver.findClassNode(ClassNodeResolver.java:168)
    groovy.util.GroovyScriptEngine$ScriptClassLoader$3.findClassNode(GroovyScriptEngine.java:218)
    org.codehaus.groovy.control.ClassNodeResolver.resolveName(ClassNodeResolver.java:124)
    org.codehaus.groovy.control.ResolveVisitor.resolveToOuter(ResolveVisitor.java:617)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromDefaultImports(ResolveVisitor.java:397)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ResolveVisitor.java:372)
    org.codehaus.groovy.control.ResolveVisitor.resolve(ResolveVisitor.java:269)
    org.codehaus.groovy.control.ResolveVisitor.visitClass(ResolveVisitor.java:1159)
    org.codehaus.groovy.control.ResolveVisitor.startResolving(ResolveVisitor.java:142)
    org.codehaus.groovy.control.CompilationUnit$11.call(CompilationUnit.java:643)
    org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:923)
    org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:585)
    org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:534)
    groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:286)
    groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:259)
    groovy.util.GroovyScriptEngine$ScriptClassLoader.doParseClass(GroovyScriptEngine.java:267)
    groovy.util.GroovyScriptEngine$ScriptClassLoader.parseClass(GroovyScriptEngine.java:236)
    groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:245)
    groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:203)
    groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:540)
    groovy.util.GroovyScriptEngine.createScript(GroovyScriptEngine.java:590)
    groovy.util.GroovyScriptEngine.run(GroovyScriptEngine.java:577)
    groovy.util.GroovyScriptEngine$run$2.call(Unknown Source)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runScript(ScriptRunnerImpl.groovy:403)
    com.onresolve.scriptrunner.runner.ScriptRunner$runScript$4.callCurrent(Unknown Source)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:145)
    com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runFileAsScript(ScriptRunnerImpl.groovy:182)
    com.onresolve.scriptrunner.runner.ScriptRunner$runFileAsScript$2.call(Unknown Source)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    com.onresolve.scriptrunner.customfield.GroovyCustomField.getValueFromIssue(GroovyCustomField.groovy:144)
    com.atlassian.jira.issue.fields.CustomFieldImpl.getValue(CustomFieldImpl.java:454)
    com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFields(SortableTextCustomFieldIndexer.java:44)
    com.atlassian.jira.issue.index.indexers.impl.SortableTextCustomFieldIndexer.addDocumentFieldsSearchable(SortableTextCustomFieldIndexer.java:33)
    com.atlassian.jira.issue.index.indexers.impl.AbstractCustomFieldIndexer.addIndex(AbstractCustomFieldIndexer.java:46)
    com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:102)
    com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAll(DefaultIssueDocumentFactory.java:91)
    com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:60)
    com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:37)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:644)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$2.perform(DefaultIssueIndexer.java:182)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$2.perform(DefaultIssueIndexer.java:175)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$3$1.get(DefaultIssueIndexer.java:334)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$3$1.get(DefaultIssueIndexer.java:330)
    com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:9)
    com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:5)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$3.consume(DefaultIssueIndexer.java:329)
    com.atlassian.jira.issue.index.DefaultIssueIndexer$3.consume(DefaultIssueIndexer.java:321)
    com.atlassian.jira.issue.util.DatabaseIssuesIterable.foreach(DatabaseIssuesIterable.java:66)
    com.atlassian.jira.issue.index.DefaultIssueIndexer.perform(DefaultIssueIndexer.java:320)
    com.atlassian.jira.issue.index.DefaultIssueIndexer.reindexIssues(DefaultIssueIndexer.java:174)
    com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.reindexUsingDatabaseLatest(DefaultIndexRecoveryManager.java:202)
    com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.reindexIssuesIn(DefaultIndexRecoveryManager.java:173)
    com.atlassian.jira.startup.IndexRecoveryLauncher.start(IndexRecoveryLauncher.java:71)
    com.atlassian.jira.startup.DefaultJiraLauncher$3.run(DefaultJiraLauncher.java:143)
    com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:345)
    com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseActivated(DatabaseConfigurationManagerImpl.java:215)
    com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:124)
    com.atlassian.jira.startup.DefaultJiraLauncher.access$100(DefaultJiraLauncher.java:33)
    com.atlassian.jira.startup.DefaultJiraLauncher$1.run(DefaultJiraLauncher.java:86)
    com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:34)
    com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:81)
    com.atlassian.jira.startup.LauncherContextListener.contextInitialized(LauncherContextListener.java:72)
    org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
    org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
    org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
    org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
    java.util.concurrent.FutureTask.run(Unknown Source)
    java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    java.lang.Thread.run(Unknown Source)

 

It seems to be related to a deadlock between HashMap and ScriptRunner.

I found the documentation for this problem, https://confluence.atlassian.com/display/JIRAKB/Deadlock+detected+on+startup+error+in+logfile, but the resolutions are not working for me. Restarting doesn't help.

To further compound the issues I can't simply role back to the previous version of JIRA as it is now incompaitble with the DB.

Any ideas of a solution?

 

3 answers

1 accepted

0 votes
Answer accepted
Mike Wells
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 18, 2015

Not too sure what the problem was, but I removed the script runner plugin from the installed-plugins folder and replaced it with the latest version. JIRA started fine after that.

0 votes
Mike Wells
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 18, 2015

I had 3.0.6 installed, not JDK8 perhaps. Anyway I removed that one and replaced it with 3.0.16. JIRA started this time. I have some fresh errors in one of my scripted JQL functions, but I suspect this is a different issue. I will open another question on that one if the issue persists.

0 votes
JamieA
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 18, 2015

That's no good... what version of the plugin? Does it happen repeatedly? If so you may have to remove it from installed-plugins, if it's blocking you from starting jira and this is not a staging instance.

Suggest an answer

Log in or Sign up to answer