Crowd XSRF Error

[Michael Genyuk]

After server reboot my crowd is throwing XSRF communication errors with my LDAP server. No changes have been made, I have tried restoring from backup but the error is still present

This is causing my entire Atlassian environment to be inaccessible

Error Message:

Test failed

There was a problem communicating with the LDAP server: XSRF check failed

the log shows this:

at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.synchroniseAll(UsnChangedCacheRefresher.java:158)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1022)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:80)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl$$Lambda$125/440580272.execute(Unknown Source)
at com.atlassian.crowd.audit.AuditLogContextInternalImpl.withAuditLogSource(AuditLogContextInternalImpl.java:52)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy85.synchronise(Unknown Source)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:85)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: levant.local:389 [Root exception is java.net.UnknownHostException: levant.local]]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.synchroniseAllGroups(UsnChangedCacheRefresher.java:194)
... 35 more
Caused by: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: levant.local:389 [Root exception is java.net.UnknownHostException: levant.local]]
at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:407)
at com.atlassian.crowd.directory.SpringLDAPConnector.searchEntitiesWithRequestControls(SpringLDAPConnector.java:440)
at com.atlassian.crowd.directory.SpringLDAPConnector.searchEntities(SpringLDAPConnector.java:424)
at com.atlassian.crowd.directory.SpringLDAPConnector.searchGroupObjectsOfSpecifiedGroupType(SpringLDAPConnector.java:1096)
at com.atlassian.crowd.directory.SpringLDAPConnector.searchGroupObjects(SpringLDAPConnector.java:1125)
at com.atlassian.crowd.directory.SpringLDAPConnector.searchGroups(SpringLDAPConnector.java:1150)
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.lambda$synchroniseAll$1(UsnChangedCacheRefresher.java:150)
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher$$Lambda$142/1399771196.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: levant.local:389 [Root exception is java.net.UnknownHostException: levant.local]]
at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:216)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:385)
at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper$3.timedGet(SpringLdapTemplateWrapper.java:154)
at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper$3.timedGet(SpringLdapTemplateWrapper.java:150)
at com.atlassian.crowd.directory.ldap.monitoring.TimedSupplier.get(TimedSupplier.java:37)
at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper.invokeWithContextClassLoader(SpringLdapTemplateWrapper.java:96)
at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper.search(SpringLdapTemplateWrapper.java:150)
at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:379)
... 11 more
Caused by: javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: levant.local:389 [Root exception is java.net.UnknownHostException: levant.local]]
at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreImpl(AbstractLdapNamingEnumeration.java:237)
at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMore(AbstractLdapNamingEnumeration.java:189)
at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:365)
... 17 more
Caused by: javax.naming.CommunicationException: levant.local:389 [Root exception is java.net.UnknownHostException: levant.local]
at com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:95)
at com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:150)
at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreReferrals(AbstractLdapNamingEnumeration.java:325)
at com.sun.jndi.ldap.AbstractLdapNamingEnumeration.hasMoreImpl(AbstractLdapNamingEnumeration.java:227)
... 19 more
Caused by: java.net.UnknownHostException: levant.local
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.sun.jndi.ldap.Connection.createSocket(Connection.java:350)
at com.sun.jndi.ldap.Connection.<init>(Connection.java:203)
at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137)
at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:64)
at com.sun.jndi.ldap.pool.Connections.<init>(Connections.java:115)
at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:132)
at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:329)
at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1605)
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2746)
at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192)
at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:151)
at com.sun.jndi.url.ldap.ldapURLContextFactory.getObjectInstance(ldapURLContextFactory.java:52)
at javax.naming.spi.NamingManager.getURLObject(NamingManager.java:601)
at javax.naming.spi.NamingManager.processURL(NamingManager.java:381)
at javax.naming.spi.NamingManager.processURLAddrs(NamingManager.java:361)
at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:333)
at com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:114)
... 22 more
11-Mar-2023 23:32:29.377 INFO [http-nio-8095-exec-3] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
2023-03-11 23:32:29,378 http-nio-8095-exec-3 INFO [server.impl.application.WebApplicationImpl] Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
11-Mar-2023 23:32:29.515 INFO [http-nio-8095-exec-3] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc
2023-03-11 23:32:29,516 http-nio-8095-exec-3 INFO [api.wadl.config.WadlGeneratorLoader] Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc
11-Mar-2023 23:32:29.517 INFO [http-nio-8095-exec-3] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor
2023-03-11 23:32:29,518 http-nio-8095-exec-3 INFO [api.wadl.config.WadlGeneratorLoader] Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor
11-Mar-2023 23:32:29.518 INFO [http-nio-8095-exec-3] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport
2023-03-11 23:32:29,518 http-nio-8095-exec-3 INFO [api.wadl.config.WadlGeneratorLoader] Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport
2023-03-11 23:32:29,603 http-nio-8095-exec-3 INFO [doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport] Found atlassian-plugin.xml file! Looking for rest plugin module descriptors...
2023-03-11 23:32:29,615 http-nio-8095-exec-3 INFO [doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport] Found 0 rest plugin module descriptors.
2023-03-11 23:32:29,668 http-nio-8095-exec-3 WARN [common.security.jersey.XsrfResourceFilter] Additional XSRF checks failed for request: http://clearmotion.org/crowd/rest/directorymanagement/1/directory/testldap/32770 , origin: http://crowd.levant.local , referrer: http://crowd.levant.local/crowd/console/secure/directory/updateconnectorconnection.action , credentials in request: true , allowed via CORS: false