Production Server Crash and is Down

[Laurie Tucker]

Enabled backup to try and create a dev instance then started the back up .. the system locked up and now we are getting

com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.NullPointerException

at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:152)

caused by: java.lang.NullPointerException

at com.atlassian.spring.container.ContainerManager.getComponent(ContainerManager.java:33)

Stack Trace: hide

com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.NullPointerException

at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:152)

at com.atlassian.util.concurrent.LazyReference.get(LazyReference.java:115)

at com.atlassian.confluence.web.filter.LanguageExtractionFilter.getLocaleManager(LanguageExtractionFilter.java:63)

at com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:48)

at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:25)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:65)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.NullPointerException

at com.atlassian.spring.container.ContainerManager.getComponent(ContainerManager.java:33)

at com.atlassian.confluence.util.LazyComponentReference$Accessor.get(LazyComponentReference.java:44)

at com.atlassian.util.concurrent.Lazy$Strong.create(Lazy.java:88)

at com.atlassian.util.concurrent.LazyReference$Sync.run(LazyReference.java:326)

at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:146)

... 25 more

[Adam Saint-Prix]

I'd shutdown JIRA and restart the application to see if it comes up. See if you can find a backup that ran previously to start your staging instance.

The errrors you've posted above don't seem to point to anything that would cause the application to hang. I suspect your logs (atlassian-jira.log) has more information in it that would be useful. You can check to see what errors appear at the time of the crash, other than the one above. Perhaps an OutOfMemoryError or something along those lines is responsible.

I'd search the jira logfile for "ERROR" and see what comes up in the results. How large is your JIRA instance? Do you normally have the XML backup running? Can you use an older backup to create the dev instance, while you sort out the application freezing issue? If not, dive into the atlasian-jira.log file and check for errors - logs don't lie.

[Laurie Tucker]

What happenned is that the services were re-started as root which then modified permissions causing the application to not re-start. Once the permissions were resolved everythign came back up. I want to thank Atlassian for their quick response in helping us resolve this issue.