Jira is not starting

Hi,

We have a self hosted instance of Jira (JIRA version = 7.4.0, Java Version = 1.8.0_102, Linux, 3.10.0-042stab125.5, started under root)

From last day, it suddenly stopped working. We tried to stop and start using stop-jira.sh/start-jira.sh. But we got an error saying something like 'the process with the PID is not found", 

So, we removed /opt/atlassian/jira/work/catalina.pid

Again we tried to restart, now it started to say, catalina.out is not accessible. So, we removed catalina.out also. 

Now it started successfully. When we try to access it , it is forwarded to a error.jsp file with 503 error. We checked the catalina.out, and found this exception

[c.a.jira.upgrade.PluginSystemLauncher] A fatal error occured during initialisation. JIRA has been locked.
com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: /var/atlassian/application-data/jira/plugins/.osgi-plugins/felix
at com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache.validate(DefaultOsgiPersistentCache.java:81)
at com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache.<init>(DefaultOsgiPersistentCache.java:38)
at com.atlassian.jira.plugin.PluginPath$JiraHomeAdapter.<init>(PluginPath.java:135)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
... 1 filtered
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:145)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:342)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:677)
at com.atlassian.jira.component.CachingMutablePicoContainer.getComponent(CachingMutablePicoContainer.java:137)
at com.atlassian.jira.ComponentManager.getComponent(ComponentManager.java:546)
at com.atlassian.jira.ComponentManager.quickStart(ComponentManager.java:194)
at com.atlassian.jira.ComponentManager.start(ComponentManager.java:165)
at com.atlassian.jira.upgrade.PluginSystemLauncher.start(PluginSystemLauncher.java:47)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDbLaunch$2(DefaultJiraLauncher.java:154)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:304)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseActivated(DatabaseConfigurationManagerImpl.java:199)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:146)
at com.atlassian.jira.startup.DefaultJiraLauncher.lambda$start$0(DefaultJiraLauncher.java:105)
at com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:31)
at com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:103)
at com.atlassian.jira.startup.LauncherContextListener.initSlowStuff(LauncherContextListener.java:157)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Failed to list contents of /var/atlassian/application-data/jira/plugins/.osgi-plugins/felix/felix-cache
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1393)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1331)
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1910)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1399)
at com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache.validate(DefaultOsgiPersistentCache.java:79)

Searched the google for this error, and found this link - https://confluence.atlassian.com/jirakb/jira-does-not-start-due-to-unable-to-clean-the-cache-directory-223219163.html

So, we removed /var/atlassian/application-data/jira/plugins/.osgi-plugins/felix and restarted the server. 

It started successfully, but when we check the issues in browser, it is displaying no issues. Only our project exists now. All the issues are erased, it seems.

Any idea how we can solve this error?

 

2 answers

2 accepted

This widget could not be displayed.

There was permission issue (we dont know what happened suddenly, it was running perfectly till yesterday)

We followed the instructions mentioned in this doc,

https://confluence.atlassian.com/jirakb/jira-applications-generate-a-filenotfoundexception-due-to-permission-denied-300810426.html

Again restarted. Again, the issues where not displayed, we could take the issues individually using direct URLs, but not through any filters. So, we did a integrity checking through admin panel. 

Then re-indexed the database using admin panel.

Now everything works fine.

Dont know what triggered these errors.

This widget could not be displayed.

This is a common error seen when you restart Confluence as the wrong user (it will work fine when you do this), then restart it again as the correct one.  It fails because the wrong user has taken ownership of files and directories such as the plugins and logs, and clobbered the permissions so that the correct owner can no longer read/write them.

So, I suspect that someone started it as root a while ago, and your restart broke when you next restarted it as the Confluence user.

Good call, Nic.  The same is true of Jira.  If other admins encounter this kind of problem in the future, it is recommended to check on the user account starting Jira and make sure that user has permissions on both the installation and home folders.  For Jira on Linux installs we have this KB to help guide on to correct this kind of problem: How to fix JIRA Directories permission in Linux

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Apr 22, 2018 in Jira Software

How-to setup a secured Jira Software 7.9.0 on Ubuntu 16.04.4 in less than 30 minutes

...PermissionsStartOnly=true User=www-data Group=www-data ExecStart=/opt/jira/bin/startup.sh ExecStop=/opt/jira/bin/shutdown.sh TimeoutStartSec=120 TimeoutStopSec=600 PrivateTmp=true [Install] WantedBy...

1,485 views 10 12
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you