nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Problems while upgrading crowd to 3.1 from 2.8

Chris Feamster

So I hope someone can help. I don't know how well this can be troubleshooted without logs, but logs don't belong in the public domain so.... :-\

I'm trying to upgrade from 2.8.8 to 3.1.1. I have a XML backup. I proceed to the step of inputting the path to the back up. After about 45s the page is reloaded with an empty error box... Additionally the follow exception is recorded after some data migrations..

2017-12-07 02:52:36,654 http-nio-8095-exec-6 INFO [atlassian.crowd.migration.XmlMigrator] Using com.atlassian.crowd.migration.GrantedPermissionMapper to import.
2017-12-07 02:52:36,676 http-nio-8095-exec-6 ERROR [console.action.setup.Import] Error importing XML data
java.util.NoSuchElementException
 at java.util.ArrayList$Itr.next(ArrayList.java:862)
 at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:302)
 at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:289)
 at com.atlassian.crowd.manager.application.InternalApplicationHelperImpl.findCrowdConsoleApplication(InternalApplicationHelperImpl.java:29)
 at com.atlassian.crowd.migration.GrantedPermissionMapper.importXml(GrantedPermissionMapper.java:75)
 at com.atlassian.crowd.migration.XmlMigrator.importXml(XmlMigrator.java:40)
 at com.atlassian.crowd.migration.XmlMigrationManagerImpl.importXml(XmlMigrationManagerImpl.java:179)
 at com.atlassian.crowd.console.action.setup.Import.update(Import.java:61)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:899)
 at ognl.OgnlRuntime.callAppropriateMethod(OgnlRuntime.java:1544)
 at ognl.ObjectMethodAccessor.callMethod(ObjectMethodAccessor.java:68)
 at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethodWithDebugInfo(XWorkMethodAccessor.java:96)
 at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethod(XWorkMethodAccessor.java:88)
 at ognl.OgnlRuntime.callMethod(OgnlRuntime.java:1620)
 at ognl.ASTMethod.getValueBody(ASTMethod.java:91)
 at ognl.SimpleNode.evaluateGetValueBody(SimpleNode.java:212)
 at ognl.SimpleNode.getValue(SimpleNode.java:258)
 at ognl.Ognl.getValue(Ognl.java:470)
 at ognl.Ognl.getValue(Ognl.java:434)
 at com.opensymphony.xwork2.ognl.OgnlUtil$3.execute(OgnlUtil.java:371)
 at com.opensymphony.xwork2.ognl.OgnlUtil.compileAndExecuteMethod(OgnlUtil.java:423)
 at com.opensymphony.xwork2.ognl.OgnlUtil.callMethod(OgnlUtil.java:369)
 at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:436)
 at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:291)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:252)
 at org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:171)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:171)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:134)
 at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:98)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:199)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:99)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:157)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.atlassian.crowd.xwork.interceptors.TransactionalInvocation.invokeAndHandleExceptions(TransactionalInvocation.java:60)
 at com.atlassian.crowd.xwork.interceptors.TransactionalInvocation.invokeInTransaction(TransactionalInvocation.java:50)
 at com.atlassian.crowd.xwork.interceptors.XWorkTransactionInterceptor.intercept(XWorkTransactionInterceptor.java:50)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.atlassian.crowd.console.interceptor.SetupCheckInterceptor.intercept(SetupCheckInterceptor.java:24)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at com.atlassian.crowd.xwork.interceptors.SecurityHeadersInterceptor.intercept(SecurityHeadersInterceptor.java:42)
 at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:247)
 at org.apache.struts2.factory.StrutsActionProxy.execute(StrutsActionProxy.java:53)
 at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:577)
 at org.apache.struts2.dispatcher.ExecuteOperations.executeAction(ExecuteOperations.java:81)
 at org.apache.struts2.dispatcher.filter.StrutsExecuteFilter.doFilter(StrutsExecuteFilter.java:88)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:125)
 at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:76)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.console.filter.CrowdDelegatingFilterProxy.doFilter(CrowdDelegatingFilterProxy.java:38)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.apache.struts2.dispatcher.filter.StrutsPrepareFilter.doFilter(StrutsPrepareFilter.java:94)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)
 at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)
 at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
 at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)
 at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.console.filter.CrowdDelegatingFilterProxy.doFilter(CrowdDelegatingFilterProxy.java:38)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.johnson.filters.AbstractJohnsonFilter.doFilter(AbstractJohnsonFilter.java:67)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.springframework.orm.hibernate5.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:151)
 at com.atlassian.crowd.console.filter.CrowdOpenSessionInViewFilter.doFilterInternal(CrowdOpenSessionInViewFilter.java:23)
 at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.plugin.web.filter.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:27)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)
 at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58)
 at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:38)
 at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39)
 at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64)
 at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70)
 at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.security.BlockOpenSocialRequestFilter.doFilter(BlockOpenSocialRequestFilter.java:39)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.console.filter.CrowdCachingFilter.doFilter(CrowdCachingFilter.java:25)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
 at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.console.filter.CrowdDelegatingFilterProxy.doFilter(CrowdDelegatingFilterProxy.java:38)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.plugin.web.filter.HttpRequestLoggingFilter.doFilter(HttpRequestLoggingFilter.java:30)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.crowd.plugin.web.filter.HttpContextFilter.doFilter(HttpContextFilter.java:30)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
 at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
 at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
 at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
 at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
 at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 at java.lang.Thread.run(Thread.java:748)
2017-12-07 02:52:53,631 http-nio-8095-exec-11 INFO [atlassian.johnson.filters.JohnsonFilter] The application is not yet setup. Redirecting request from '/console/login.action' to '
/console/setup/selectsetupstep.action'
2017-12-07 02:52:53,680 http-nio-8095-exec-10 INFO [console.action.setup.SelectSetupStep] Current setup step is /console/setup/setupimport.action

This exception appears generic to me.

Why would a backup that worked just moments ago fail during this upgrade. Are there manual data/model steps that need to be executed?

Any and all suggestions welcome.

Thank you.

Accepted answers

lpater

For reference, the issue turned out to be caused by not setting the MySQL transaction isolation level to READ-COMMITTED, as documented here.

I've raised CWD-5007 to track the issue. The interim solution is to set the transaction isolation level appropriately.

All comments

lpater

Hi Chris,

It seems that the backup you're trying to restore doesn't contain an application of type CROWD, which is expected by the import process.

This generally shouldn't be the case, but might potentially might be caused by some manual changes in the database in the past, or some unexpected migration path in the past.

Please double-check your backup xml, to check if the Crowd application is there, and has the type CROWD. It should look like this:

...
 <application>
 <id>1234</id>
 <name>crowd</name>
...
 <active>true</active>
 <type>CROWD</type>
...

If the crowd application has a different type, please try changing it back to <type>CROWD</type> (for example <type>GENERIC_APPLICATION</type>, and trying to reimport the backup again.

Hope this helps, let us know if you run into any other issues.

Chris Feamster

@lpater Thanks for the lead!

Unfortunately that didn't go anywhere:

 <applications>
  <application>
   <id>2</id>
   <name>crowd</name>
   <createdDate>Fri Jan 03 02:44:41 +0000 2014</createdDate>
   <updatedDate>Sun May 10 05:26:34 +0000 2015</updatedDate>
   <active>true</active>
   <type>CROWD</type>

If not that, then what?

Marcin Kempa

Hi @Chris Feamster,

You mentioned that the 'backup worked just a moments ago', does that mean that you tested it on some previous Crowd version or was it just generated a moments ago on your 2.8.8 instance?

Do you see some other errors in the log file that preceded the one you've already mentioned, especially something like:

[ERROR] Could not add <name of application>

Could you also try to use this backup on the 2.8.8 Crowd version in your test environment and share with us the results?

Thanks,

Marcin Kempa

Chris Feamster

Hi @Marcin Kempa,

I meant that it was just generated moments ago. Indeed, technically this is a DR scenario. I restored a week-old backup to 3.1. It failed. I restored that back up to 2.8.8. It succeeded. I made a new backup yesterday to upgrade to 3.1.1, It failed (the cause of this ticket).

So actually this day old backup is the result of a successful restore of a previous week old backup. So right now it seems as if it is 3.1.1 specifically that is incompatible with the data from my installation.

Thoughts?

(I'd be great if there was a secure way to send you a sql dump... )

AnnWorley

Chris, you should get an email from our support portal shortly. We created a support request in your name so you could use the file sharing functionality to provide logs and a backup, etc. for Lukasz to examine.

lpater

For reference, the issue turned out to be caused by not setting the MySQL transaction isolation level to READ-COMMITTED, as documented here.

I've raised CWD-5007 to track the issue. The interim solution is to set the transaction isolation level appropriately.

Rob Cascella

What is the workaround for this in cases where one cannot configure the MySQL transaction isolation level, for example in Amazon RDS?

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

import

crowd

upgrade

database