Unexpected exception during JIRA startup. This JIRA instance will not be able to recover. Please che

Katie June 20, 2020

Hi Community,

 

We have been struggling with our Jira access for a few weeks. Our instance is located in our own server. We tried to stop and restart the server but it keeps showing a fatal level problem with the following description: "Unexpected exception during JIRA startup. This JIRA instance will not be able to recover. Please check the logs for details".

We checked the logs and still have no idea how to recover our jira server. Any help will be highly appreciated.

Caused by java.1ang.IllegalArgumentxnd in action.pngScreen Shot 2020-06-20 at 16.44.54.png

 

 

3 answers

0 votes
Najjar _Innovura_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
July 15, 2020

Restart the server to close and delete all relevant cache

0 votes
Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 20, 2020

You'll need to read the rest of the errors in the log around this one - it's only part of the problem and doesn't tell us enough.

Did you change anything since you last had a running Jira?  I ask because this sort of error is usually structural, so if you've upgraded Jira or changed the version of Java it is using, you can get it.

Katie June 20, 2020

Thanks for your help! We didn't change anything for a long time. our server version is 7.3.6. 

Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 20, 2020

Ok, that rules a few things out.  What do the rest of the logs say?

Katie June 20, 2020

20-Jun-2020 02:57:35.045 SEVERE [http-nio-8080-exec-13] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=404, location=/secure/404.jspa]

javax.servlet.ServletException: java.lang.IllegalArgumentException: No command 'Is-Struts2-Vul-URL' in action

at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.onActionException(JiraWebworkActionDispatcher.java:226)

at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(JiraWebworkActionDispatcher.java:178)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.jira.web.filters.XContentTypeOptionsNoSniffFilter.doFilter(XContentTypeOptionsNoSniffFilter.java:20)

at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:28)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsCommonFilter.preventDoubleInvocation(PrettyUrlsCommonFilter.java:38)

at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter.doFilter(PrettyUrlsSiteMeshFixupFilter.java:22)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:55)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsCommonFilter.preventDoubleInvocation(PrettyUrlsCommonFilter.java:38)

at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:65)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:51)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)

at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)

at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)

at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:74)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.jira.web.filters.MultipartBoundaryCheckFilter.doFilter(MultipartBoundaryCheckFilter.java:36)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:74)

at com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(JiraFirstFilter.java:59)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at com.atlassian.gzipfilter.GzipFilter.doFilterInternal(GzipFilter.java:115)

at com.atlassian.gzipfilter.GzipFilter.doFilter(GzipFilter.java:92)

at com.atlassian.jira.web.filters.gzip.JiraGzipFilter.doFilter(JiraGzipFilter.java:44)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)

at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:726)

at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:471)

at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:394)

at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:311)

at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:395)

at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:254)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:177)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)

at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)

at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:784)

at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)

at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:802)

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410)

at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.IllegalArgumentException: No command 'Is-Struts2-Vul-URL' in action

at webwork.action.ActionSupport.invokeCommand(ActionSupport.java:429)

at webwork.action.ActionSupport.execute(ActionSupport.java:157)

at com.atlassian.jira.action.JiraActionSupport.execute(JiraActionSupport.java:63)

at webwork.interceptor.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:39)

at webwork.interceptor.NestedInterceptorChain.proceed(NestedInterceptorChain.java:31)

at webwork.interceptor.ChainedInterceptor.intercept(ChainedInterceptor.java:16)

at webwork.interceptor.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:35)

at webwork.dispatcher.GenericDispatcher.executeAction(GenericDispatcher.java:225)

at webwork.dispatcher.GenericDispatcher.executeAction(GenericDispatcher.java:154)

at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(JiraWebworkActionDispatcher.java:147)

... 95 more

 

That's basically all the thing I can see from log.

Katie June 20, 2020

[root@jira logs]# vi catalina.2020-06-20.log

 

it A valid shutdown command was received via the shutdown port. Stopping the Server instance.

20-Jun-2020 11:38:52.976 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8080"]

20-Jun-2020 11:38:53.027 INFO [main] org.apache.catalina.core.StandardService.stopInternal Stopping service Catalina

20-Jun-2020 11:39:11.436 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appech] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:502)

org.objectweb.jonas_timer.TimerManager.batch(TimerManager.java:193)

org.objectweb.jonas_timer.Batch.run(TimerManager.java:65)

20-Jun-2020 11:39:11.436 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appeck] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

java.lang.Thread.sleep(Native Method)

org.objectweb.jonas_timer.TimerManager.clock(TimerManager.java:142)

org.objectweb.jonas_timer.Clock.run(TimerManager.java:46)

Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 20, 2020

That is Tomcat recording that someone or something asked it to stop (and failing on part of it)

We need to be looking all of the errors you get from an attempt to start it.

I would stop it all, and move all the current logs out to a backup location, so it will generate clean ones with nothing but the relevant logging in them. 

The logs are in

<jira home>/log

<jira install>/log

After cleaning and restarting, look at all the errors in <jira home>/log/atlassian-jira.log and if they don't tell us anything, <jira install>/log/catalina.out

Katie June 20, 2020

Screen Shot 2020-06-20 at 20.55.00.pngScreen Shot 2020-06-20 at 20.55.13.pngScreen Shot 2020-06-20 at 20.55.38.pngScreen Shot 2020-06-20 at 20.55.24.pngScreen Shot 2020-06-20 at 20.55.47.png

Katie June 20, 2020

The error log is kind of long and It stuck somewhere. 

Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 20, 2020

I can't open these as pictures, could you give us the actual text?

But, before you spend time on that, from what I could see from those pictures, I noticed one possible problem.  What I can see tells us that the permissions are wrong.

Could you stop your Jira, make sure your install and data directories for Jira are all owned by the account that is going to run it, and then restarrt it?

Najjar _Innovura_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
June 21, 2020

It seems Jira is trying to access its folder a but failing with permission denied error thrown. 

If you are running Jira as a Windows Service, try to use an administrator account as the logged in user of the service

If you are running Jira on Linux then you need to modify Jira installation directory and home permissions using chmod

I always like to try with chmod 777 -R * on both folders and try to restart Jira 

Najjar _Innovura_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
June 21, 2020

Also it seems that there is another instance of Jira running on the same server, where it throws another error saying that Jira home is being used with another Jira instance,

And when jira tries to start its tomcat server it crashes with Port already in use error.

 

Could you try the following command to check if there's another instance running

ps -ef | grep jira

Katie June 21, 2020

[root@jira ~]# ps -ef | grep jira

root      6022  5973  0 18:57 pts/0    00:00:00 grep jira

jira     29447     1  0 Jun20 ?        00:00:58 /opt/atlassian/jira/jre//bin/java -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx1024m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -Datlassian.plugins.enable.wait=300 -XX:+PrintGCDateStamps -XX:-OmitStackTraceInFastThrow -Datlassian.plugins.startup.options= -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -classpath /opt/atlassian/jira/bin/bootstrap.jar:/opt/atlassian/jira/bin/tomcat-juli.jar -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp org.apache.catalina.startup.Bootstrap start

Najjar _Innovura_
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
June 21, 2020

The log file is showing that jira is already running on this machine, 

Can you to access it from browser?

Katie June 22, 2020

JIRA Startup Failed

Unable to clean the cache directory: /var/atlassian/application-data/jira/plugins/.osgi-plugins/felix caused by: Failed to list contents of /var/atlassian/application-data/jira/plugins/.osgi-plugins/felix/felix-cache

Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 22, 2020

This is one of the symptoms of having the wrong permissions on the installation and/or data directories.

Could you stop your Jira, make sure your install and data directories for Jira are all owned by the jira account that is going to run it, and then restarrt it?

Katie July 15, 2020

we tried changing permissions and it still gives me same error.

0 votes
Mathis Hellensberg
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 20, 2020

Hi @Katie

Have you tried enabling Debugging for your instance? It might help you get more information about what is causing the issue. 

You could also try starting up in safe mode :)

Suggest an answer

Log in or Sign up to answer