HTTP 500 error after upgrading atlassian plugin

[Tom Cassidy]

I saw an error popup when updating a plugin and now Confluence is giving me a HTTP 500 error.

I have tried clearing the plugin cache, restarting confluence, and running the installer again with the update existing server option but the problem remains.

From `atlassian-confluence.log`: 

2023-01-12 02:15:00,028 INFO [Caesium-1-3] [atlassian.confluence.event.MonitorableCallerRunsPolicy] rejectedExecution Asynchronous queue is full. The task will be processed synchronously in the current thread instead. We'll print this message every 60 seconds until there's enough space in queue to start processing messages asynchronously again. Executor: [java.util.concurrent.ThreadPoolExecutor@2a913324[Running, pool size = 2, active threads = 2, queued tasks = 143, completed tasks = 34847]]
2023-01-12 08:52:29,300 INFO [Caesium-1-1] [ratelimiting.internal.configuration.ConfigurationLoggerJob] logConfiguration Periodic rate limiting configuration log. System rate limiting settings: [SystemRateLimitingSettings(mode=OFF, bucketSettings=TokenBucketSettings(capacity=50, fillRate=10, intervalFrequency=1, intervalTimeUnit=Seconds), jobControlSettings=SystemJobControlSettings(reportingDbArchivingJobFrequencyDuration=PT1M10S, reportingDbRetentionPeriodDuration=PT24H, bucketCollectionJobFrequencyDuration=PT5M, bucketCleanupJobFrequencyDuration=PT15M, settingsReloadJobFrequencyDuration=PT30S))]
2023-01-12 10:11:09,385 WARN [http-nio-127.0.0.1-8090-exec-9 url: /rest/plugins/1.0/installed-marketplace; user: myusername] [atlassian.upm.pac.PacClientImpl] getUpdates The request to check for app updates may take longer than expected because 94 apps are installed that could have updates. This may impact the performance of loading the Manage Apps page.
 -- url: /rest/plugins/1.0/installed-marketplace | userName: myusername | referer: https://wiki.example.com/plugins/servlet/upm | traceId: 845c300cd0a30214
2023-01-12 10:12:06,016 INFO [UpmAsynchronousTaskManager:thread-3] [atlassian.plugin.manager.DefaultPluginManager] lambda$updatePlugin$26 Updating plugin 'com.atlassian.plugins.authentication.atlassian-authentication-plugin' from version '4.2.17' to version '4.2.20'
2023-01-12 10:12:06,018 INFO [UpmAsynchronousTaskManager:thread-3] [atlassian.plugin.manager.DefaultPluginManager] lambda$broadcastPluginDisabling$39 Disabling com.atlassian.plugins.authentication.atlassian-authentication-plugin
2023-01-12 10:12:06,889 ERROR [lifecycle:thread-27] [sal.confluence.lifecycle.TenantAwareLifecycleManager] triggerLifecycleAsTenant Unable to start component: com.atlassian.plugins.authentication.impl.web.oidc.OidcDiscoveryRefreshJob
 -- url: /rest/plugins/1.0/ | userName: myusername | referer: https://wiki.example.com/plugins/servlet/upm | traceId: 14812658c7a830cf
java.lang.RuntimeException: java.lang.IllegalStateException: plugin [{com.atlassian.plugins.authentication.atlassian-authentication-plugin}] invoking ActiveObjects before <ao> configuration module is enabled or plugin is missing an <ao> configuration module. Note that scanning of entities from the ao.model package is no longer supported.
	at com.atlassian.plugins.authentication.impl.web.oidc.OidcDiscoveryRefreshJob.onStart(OidcDiscoveryRefreshJob.java:87)
	at com.atlassian.sal.confluence.lifecycle.TenantAwareLifecycleManager$PerTenantLifecycleExecution.lambda$triggerLifecycleAsTenant$1(TenantAwareLifecycleManager.java:197)
	at com.atlassian.confluence.tenant.VolatileSingleTenantRegistry.asTenant(VolatileSingleTenantRegistry.java:49)
	at jdk.internal.reflect.GeneratedMethodAccessor389.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
	at com.sun.proxy.$Proxy663.asTenant(Unknown Source)
	at jdk.internal.reflect.GeneratedMethodAccessor389.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at com.sun.proxy.$Proxy998.asTenant(Unknown Source)
	at com.atlassian.sal.confluence.lifecycle.TenantAwareLifecycleManager$PerTenantLifecycleExecution.triggerLifecycleAsTenant(TenantAwareLifecycleManager.java:195)
	at com.atlassian.sal.confluence.lifecycle.TenantAwareLifecycleManager$PerTenantLifecycleExecution.lambda$addLifecycles$0(TenantAwareLifecycleManager.java:186)
	at com.atlassian.sal.confluence.lifecycle.StaticServiceExecution.execute(StaticServiceExecution.java:23)
	at com.atlassian.sal.confluence.lifecycle.ServiceExecutionTemplate.call(ServiceExecutionTemplate.java:48)
	at com.atlassian.sal.confluence.lifecycle.ServiceExecutionTemplate.call(ServiceExecutionTemplate.java:20)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: plugin [{com.atlassian.plugins.authentication.atlassian-authentication-plugin}] invoking ActiveObjects before <ao> configuration module is enabled or plugin is missing an <ao> configuration module. Note that scanning of entities from the ao.model package is no longer supported.
	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.delegate(TenantAwareActiveObjects.java:168)
	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.executeInTransaction(TenantAwareActiveObjects.java:349)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at com.sun.proxy.$Proxy3295.executeInTransaction(Unknown Source)
	at com.atlassian.activeobjects.tx.TransactionalProxy.executeInTransaction(TransactionalProxy.java:45)
	at com.atlassian.activeobjects.tx.TransactionalProxy.invokeInTransaction(TransactionalProxy.java:38)
	at com.atlassian.activeobjects.tx.TransactionalProxy.invoke(TransactionalProxy.java:30)
	at com.sun.proxy.$Proxy3324.getSsoConfig(Unknown Source)
	at com.atlassian.plugins.authentication.impl.web.oidc.OidcDiscoveryRefreshJob.getCronFromConfig(OidcDiscoveryRefreshJob.java:122)
	at com.atlassian.plugins.authentication.impl.web.oidc.OidcDiscoveryRefreshJob.scheduleJobForCronInterval(OidcDiscoveryRefreshJob.java:109)
	at com.atlassian.plugins.authentication.impl.web.oidc.OidcDiscoveryRefreshJob.onStart(OidcDiscoveryRefreshJob.java:85)
	... 36 more
2023-01-12 10:12:07,611 ERROR [http-nio-127.0.0.1-8090-exec-2] [[Standalone].[localhost].[/].[noop]] log Servlet.service() for servlet [noop] in context with path [] threw exception [Filter execution threw an exception] with root cause
java.lang.ClassNotFoundException: com.opensymphony.xwork.config.ConfigurationManager not found by com.atlassian.plugins.authentication.atlassian-authentication-plugin [302]
	at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1585)
	at org.apache.felix.framework.BundleWiringImpl.access$300(BundleWiringImpl.java:79)
	at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1970)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at com.atlassian.plugins.authentication.impl.web.filter.authentication.ConfluenceAuthenticationFilter.shouldFilter(ConfluenceAuthenticationFilter.java:78)
	at com.atlassian.plugins.authentication.impl.web.filter.authentication.ConfluenceAuthenticationFilter.doFilterInternal(ConfluenceAuthenticationFilter.java:85)
	at com.atlassian.plugins.authentication.impl.web.filter.AbstractJohnsonAwareFilter.doFilter(AbstractJohnsonAwareFilter.java:29)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:62)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.lambda$doFilter$0(DelegatingPluginFilter.java:57)
	at com.atlassian.confluence.plugins.pulp.PulpFilter.doFilter(PulpFilter.java:75)
	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:62)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)
...

2023-01-12 10:12:08,894 INFO [http-nio-127.0.0.1-8090-exec-2 url: /500page.jsp; user: myusername] [atlassian.confluence.status.SystemErrorInformationLogger] writeToLog 
Request Unique ID : ca1ef807-9a8f-45c7-8d42-46a02b85033c
--------------------------
JVM Stats
--------------------------
xmx = 1073741824
usedNonHeap = 566473592
availableHeap = 443696496
availableNonHeap = -1
allocatedHeap = 1073741824
freeAllocatedHeap = 443696496
maxNonHeap = -1
availablePermGen = 0
maxPermGen = -1
maxHeap = 1073741824
usedHeap = 630045328
usedPermGen = -1
xms = 1073741824
--------------------------
Request Information
--------------------------
URL: https://wiki.example.com/500page.jsp
Scheme: https
Server: wiki.example.com
Port: 443
URI: /500page.jsp
Context Path: 
Servlet Path: /500page.jsp
Path Info: null
Query String: _=1673478666952
--------------------------
Attributes
--------------------------
javax.servlet.forward.request_uri: /rest/plugins/1.0/pending/49d1e914-b9b5-4769-a8c7-23cfba64f1de
javax.servlet.forward.context_path: 
javax.servlet.forward.servlet_path: /rest
javax.servlet.forward.path_info: /plugins/1.0/pending/49d1e914-b9b5-4769-a8c7-23cfba64f1de
javax.servlet.forward.query_string: _=1673478666952
javax.servlet.forward.mapping: org.apache.catalina.core.ApplicationMapping$MappingImpl@57a4dfaa
org.apache.catalina.AccessLog.RemoteAddr: xxx.xxx.xxx.xxx
__prepare_recursion_counter: 1
javax.servlet.error.status_code: 500
struts.actionMapping: noActionMapping
brave.SpanCustomizer: SpanCustomizer(RealSpan(382d265c7fa52766/382d265c7fa52766))
com.atlassian.confluence.impl.profiling.DecoratorTimings: com.atlassian.confluence.impl.profiling.DecoratorTimings@4a213dc6
org.apache.struts2.dispatcher.filter.StrutsPrepareFilter.REQUEST_EXCLUDED_FROM_ACTION_MAPPING: false
com.opensymphony.sitemesh.APPLIED_ONCE: true
__wrap_recursion_counter: 1
com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter_already_filtered: true
atlassian.core.seraph.original.url: /500page.jsp?_=1673478666952
org.apache.catalina.AccessLog.Protocol: HTTP/1.1
com.atlassian.gzipfilter.GzipFilter_already_filtered: true
com.atlassian.seraph.auth.LoginReason: OK
org.apache.catalina.AccessLog.ServerPort: 443
javax.servlet.error.message: 
javax.servlet.error.servlet_name: noop
org.apache.tomcat.request.forwarded: true
brave.propagation.TraceContext: 382d265c7fa52766/382d265c7fa52766
brave.servlet.TracingFilter$SendHandled: true
org.apache.tomcat.remoteAddr: xxx.xxx.xxx.xxx
org.apache.catalina.AccessLog.ServerName: wiki.example.com
B3-TraceId: 7d30f20c43234c
loginfilter.already.filtered: true
javax.servlet.error.request_uri: /rest/plugins/1.0/pending/49d1e914-b9b5-4769-a8c7-23cfba64f1de
com.atlassian.core.filters.HeaderSanitisingFilter_already_filtered: true
com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter: true
org.apache.catalina.AccessLog.RemoteHost: xxx.xxx.xxx.xxx
javax.servlet.error.exception: javax.servlet.ServletException: Filter execution threw an exception
os_securityfilter_already_filtered: true
com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter: true
--------------------------
Parameters
--------------------------
_ : 1673478666952
caused by: javax.servlet.ServletException: Filter execution threw an exception
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:196)
caused by: java.lang.NoClassDefFoundError: com/opensymphony/xwork/config/ConfigurationManager
at com.atlassian.plugins.authentication.impl.web.filter.authentication.ConfluenceAuthenticationFilter.shouldFilter(ConfluenceAuthenticationFilter.java:78)
caused by: java.lang.ClassNotFoundException: com.opensymphony.xwork.config.ConfigurationManager not found by com.atlassian.plugins.authentication.atlassian-authentication-plugin [302]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1585)

2023-01-12 10:12:08,907 WARN [UpmScheduler:thread-2] [atlassian.upm.pac.PacClientImpl] getUpdates The request to check for app updates may take longer than expected because 94 apps are installed that could have updates. This may impact the performance of loading the Manage Apps page.
2023-01-12 10:12:09,528 ERROR [http-nio-127.0.0.1-8090-exec-11] [[Standalone].[localhost].[/].[servlet-module-container-servlet]] log Servlet.service() for servlet [servlet-module-container-servlet] in context with path [] threw exception [Filter execution threw an exception] with root cause
java.lang.NoClassDefFoundError: com/opensymphony/xwork/config/ConfigurationManager
	at com.atlassian.plugins.authentication.impl.web.filter.authentication.ConfluenceAuthenticationFilter.shouldFilter(ConfluenceAuthenticationFilter.java:78)
	at com.atlassian.plugins.authentication.impl.web.filter.authentication.ConfluenceAuthenticationFilter.doFilterInternal(ConfluenceAuthenticationFilter.java:85)
	at com.atlassian.plugins.authentication.impl.web.filter.AbstractJohnsonAwareFilter.doFilter(AbstractJohnsonAwareFilter.java:29)
...

 How can I fix this?

[Cedric Stanislas · Accepted solution]

Hi @Tom Cassidy 

We've just ran into the same issue.

The v4.2.20 of the com.atlassian.plugins.authentication.atlassian-authentication-plugin seems to have some compatibility we directly changed this pluging the the database in order to fix the issue

In the Table PLUGINDATA you should try to delete the line which contains : com.atlassian.plugins.authentication.atlassian-authentication-plugin and then start confluence. 

Let me know if that helped you.

[Tom Cassidy]

That has got it back online, thanks!

delete from plugindata where pluginkey = 'com.atlassian.plugins.authentication.atlassian-authentication-plugin';