Cannot install or upgrade any plugins in preprod instance

[Jacob M. Dawson]

All attempts to upgrade or install plugins fail. It appears that UPM is not downloading them correctly, as the file in $CONF_DATA/plugins-cache is larger than when I use wget to get the same file. I get exactly the same result when I attempt to get the file from URL or from 'upload.' Had the same problems when updating UPM from 2.0.6, but managed to use the manual update methods, including extracting atlassian-bundled-plugins.zip and replacing the jar, to achieve the upgrade. Problem still affects other plugins.

Short form of errors

2013-02-12 11:51:05,828 ERROR pool-3-thread-3 atlassian.plugin.loaders.ScanningPluginLoader deployPluginFromUnit Unable to deplo

y plugin 'null', file Unit: /var/local/confluence-data/plugins-cache/1360687865780plugin_1473590140481235282_confluence-source-edito

r-1.0.6.jar (1360687865000)

-- referer: https://rdweb-preprod.cns.vt.edu/wiki/plugins/servlet/upm/marketplace | url: /wiki/rest/plugins/1.0/available/featured

| userName: dawson

com.atlassian.plugin.PluginParseException: Cannot open JAR file for reading: /var/local/confluence-data/plugins-cache/1360687865780p

lugin_1473590140481235282_confluence-source-editor-1.0.6.jar

at com.atlassian.plugin.JarPluginArtifact.getResourceAsStream(JarPluginArtifact.java:60)

at com.atlassian.plugin.factories.LegacyDynamicPluginFactory.canCreate(LegacyDynamicPluginFactory.java:137)

...

2013-02-12 11:51:05,843 ERROR pool-3-thread-3 com.atlassian.upm.PluginInstaller execute Failed to install plugin

-- referer: https://rdweb-preprod.cns.vt.edu/wiki/plugins/servlet/upm/marketplace | url: /wiki/rest/plugins/1.0/available/featured

| userName: dawson

com.atlassian.plugin.PluginException: Plugin failed to install: plugin_1473590140481235282_confluence-source-editor-1.0.6.jar

at com.atlassian.upm.PluginAccessorAndControllerImpl$1.doInTransaction(PluginAccessorAndControllerImpl.java:153)

at com.atlassian.upm.PluginAccessorAndControllerImpl$1.doInTransaction(PluginAccessorAndControllerImpl.java:124)

....

2013-02-12 11:51:05,986 WARN pool-3-thread-3 rest.resources.install.InstallTask call Unexpected error in install task

-- referer: https://rdweb-preprod.cns.vt.edu/wiki/plugins/servlet/upm/marketplace | url: /wiki/rest/plugins/1.0/available/featured

| userName: dawson

java.lang.RuntimeException: com.atlassian.plugin.PluginException: Plugin failed to install: plugin_1473590140481235282_confluence-so

urce-editor-1.0.6.jar

at com.atlassian.upm.PluginInstaller.execute(PluginInstaller.java:202)

at com.atlassian.upm.PluginInstaller.install(PluginInstaller.java:103)

at com.atlassian.upm.rest.resources.install.InstallTask.installFromFile(InstallTask.java:110)

at com.atlassian.upm.rest.resources.install.InstallFromFileTask.executeTask(InstallFromFileTask.java:33)

...

Caused by: com.atlassian.plugin.PluginException: Plugin failed to install: plugin_1473590140481235282_confluence-source-editor-1.0.6

.jar

....

[Vishnukumar Vasudevan]

Can you try the below solution ?

1. Download jar files of all required plugins.

2. shut down JIRA

3. Remove old versions of plugins from <JIRA_Home>plugins/installed-plugins/

4. Copy new jar files to the above path

5. start JIRA.

This should upgrade plugins over the start-up. Recommend to keep back up of JIRA and home directory as well.

Thanks, Vishnu.

[Jacob M. Dawson]

There's no plugins directory in $CONF_DATA, or anything roughly equivalent. There are several variants on plugins-cache, and bundled-plugins, but that's it. I've already tried removing everything from the plugindata table, with no success.

[Nic Brough -Adaptavist-]

I'm afraid Vishnu misled you and spoke about Jira instead of Confluence.

Right idea though - killing off all the non-core plugins is worth a try. To do that, stop Confluence, MOVE all the directories relating to plugins-cache somewhere safe, and restart it. (if you move it, you can put it back to do more testing later). Do not remove bundled-plugins.

Removing the data from the plugindata table may well have broken it badly though.

[Jacob M. Dawson]

Deleting everything from plugindata is one of the more drastic steps they mention, but it hasn't broken anything. I've already cleared all the plugins-chache directories. Bundled-plugins gets rebuilt when you restart the application, anyway, so removing things from it is fairly pointless. At worst, I've busted all the user-installed plugins, but those can be recovered once I get the thing to the point where I can install plugins again. Right now, it just fails very quickly.

[Vishnukumar Vasudevan]

Jacob, I am so sorry. The steps I mentioned was for JIRA. Though you can make a try as Nic said.

Nic - Thanks a lot for correcting me. The question was not taged properly and I didn't notice the log entries were pointing to Confluence :(

[Nic Brough -Adaptavist-]

@Vishnu :-) I do it too. You can't see much evidence because I have an annoying habit of re-reading stuff because my writing is awful and usually needs editing! You still had exactly the right idea - forcibly killing off the plugins is the first thing to try.

@Jacob Dawson : So, after you kill off all the plugin cache directories, does that change the error messages?

[Jacob M. Dawson]

Nic Brough Nope, this is the result after I've already done that.

[Nic Brough -Adaptavist-]

Hmm. I'm a bit stuck then. I don't know where it's pulling the attempts to load the plugins from if you've cleared them out of the database and removed the physical installs.

My instinct is to start with a clean install of the same version (without any plugins of course), point it at the database after you've proved you can get a clean startup out of it.

[Jacob M. Dawson]

At this point, I'm just waiting for closure from Atlassian before I clear it all out and start with a fresh copy of the production instance, see where it goes from there. The problem I'm having at this point isn't with updating user-added plugins, but the bungled plugins, or adding new plugins that I don't already have. All of these operations fail instantly with the above errors. Frankly, it doesn't look like it's taking long enough to actually retrieve the plugins in question, and given that the files are coming out the wrong size in the filesystem, that appears to be where the problem is.