Hi,
After upgrading Confluence from 5.10.8 to 6.5.2, I'm facing issues with the LDAP directory synchronisation. Here are the errors I have in the logs:
2018-03-02 15:44:50,719 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteDirectory] synchroniseCache INCREMENTAL synchronisation for directory [ 220037125 ] starting
2018-03-02 15:44:50,720 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteDirectory] synchroniseCache Attempting INCREMENTAL synchronisation for directory [ 220037125 ]
2018-03-02 15:44:50,720 INFO [Caesium-1-4] [directory.ldap.cache.UsnChangedCacheRefresher] synchroniseChanges After restarting, full sync of directory [220037125] is necessary before incremental sync is possible.
2018-03-02 15:44:50,721 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteDirectory] synchroniseCache Incremental synchronisation for directory [ 220037125 ] was not completed, falling back to a full synchronisation
2018-03-02 15:44:50,721 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteDirectory] synchroniseCache INCREMENTAL synchronisation for directory [ 220037125 ] was not successful, attempting FULL
2018-03-02 15:44:53,574 INFO [CrowdUsnChangedCacheRefresher:thread-2] [crowd.directory.ldap.SpringLdapTemplateWrapper] call Timed call for search with handler on dc=ubisoft,dc=org took 2832ms
2018-03-02 15:45:13,701 INFO [CrowdUsnChangedCacheRefresher:thread-1] [crowd.directory.ldap.SpringLdapTemplateWrapper] call Timed call for search with handler on dc=ubisoft,dc=org took 22953ms
2018-03-02 15:45:35,631 INFO [CrowdUsnChangedCacheRefresher:thread-1] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 27453 ] remote users in [ 44909ms ]
2018-03-02 15:45:37,861 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned and compared [ 27453 ] users for delete in DB cache in [ 2174ms ]
2018-03-02 15:45:37,861 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned for deleted users in [ 2174ms ]
2018-03-02 15:45:39,508 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] getUsersToAddAndUpdate scanning [ 27453 ] users to add or update
2018-03-02 15:45:39,578 INFO [Caesium-1-4] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers scanned and compared [ 27453 ] users for update in DB cache in [ 1539ms ]
2018-03-02 15:45:39,579 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] updateUsers updating [ 385 ] users
2018-03-02 15:45:40,655 INFO [CrowdUsnChangedCacheRefresher:thread-2] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 68331 ] remote groups in [ 49933ms ]
2018-03-02 15:45:41,365 ERROR [Caesium-1-4] [atlassian.confluence.event.ConfluenceEventDispatcher] run There was an exception thrown trying to dispatch event[com.atlassian.confluence.event.events.user.DirectoryUserRenamedEvent[source=com.atlassian.confluence.user.crowd.CachedCrowdUserDao@5d7ff65d]] from the invoker[com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@3d7964b]
java.lang.RuntimeException: java.lang.NullPointerException
at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:39)
at com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1.invoke(ConfluenceListenerHandlersConfiguration.java:69)
at com.atlassian.confluence.event.ConfluenceEventDispatcher$VCacheRequestContextRunnableFactory$1.lambda$run$0(ConfluenceEventDispatcher.java:93)
at com.atlassian.confluence.vcache.VCacheRequestContextOperations.lambda$doInRequestContext$0(VCacheRequestContextOperations.java:50)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
at com.atlassian.confluence.vcache.VCacheRequestContextOperations.doInRequestContext(VCacheRequestContextOperations.java:49)
at com.atlassian.confluence.event.ConfluenceEventDispatcher$VCacheRequestContextRunnableFactory$1.run(ConfluenceEventDispatcher.java:93)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:88)
at com.atlassian.event.internal.EventPublisherImpl.invokeListeners(EventPublisherImpl.java:203)
at com.atlassian.event.internal.EventPublisherImpl.publish(EventPublisherImpl.java:114)
at com.atlassian.confluence.event.TimingEventPublisher.publish(TimingEventPublisher.java:64)
at com.atlassian.confluence.user.crowd.CachedCrowdUserDao.rename(CachedCrowdUserDao.java:231)
at sun.reflect.GeneratedMethodAccessor973.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at com.atlassian.spring.interceptors.SpringProfilingInterceptor.invoke(SpringProfilingInterceptor.java:16)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy91.rename(Unknown Source)
at com.atlassian.crowd.directory.AbstractInternalDirectory.forceRenameUser(AbstractInternalDirectory.java:661)
at com.atlassian.crowd.directory.DbCachingRemoteChangeOperations.updateUsers(DbCachingRemoteChangeOperations.java:246)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.atlassian.crowd.directory.$Proxy2592.updateUsers(Unknown Source)
at com.atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations.addOrUpdateCachedUsers(DirectoryCacheImplUsingChangeOperations.java:65)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllUsers(UsnChangedCacheRefresher.java:190)
at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseAll(AbstractCacheRefresher.java:56)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAll(UsnChangedCacheRefresher.java:163)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1186)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:74)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:50)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:96)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.doRunJob(JobRunnerWrapper.java:120)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.lambda$runJob$0(JobRunnerWrapper.java:91)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.runJob(JobRunnerWrapper.java:91)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeLocalJob(CaesiumSchedulerService.java:410)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:388)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.atlassian.confluence.user.persistence.dao.hibernate.HibernateConfluenceUserDao.rename(HibernateConfluenceUserDao.java:142)
at com.atlassian.confluence.impl.user.persistence.dao.CachingConfluenceUserDao.rename(CachingConfluenceUserDao.java:63)
at sun.reflect.GeneratedMethodAccessor977.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy71.rename(Unknown Source)
at com.atlassian.confluence.user.listeners.UserDirectoryListener.onUserRenamedEvent(UserDirectoryListener.java:67)
at sun.reflect.GeneratedMethodAccessor975.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:32)
... 67 more
2018-03-02 15:45:41,556 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] updateUsers updated [ 385 ] users successfully in [ 1977ms ]
2018-03-02 15:45:41,557 WARN [Caesium-1-4] [confluence.impl.hibernate.ConfluenceHibernateTransactionManager] doRollback Performing rollback. Transactions:
->[com.atlassian.crowd.directory.DbCachingRemoteChangeOperations.updateUsers]: PROPAGATION_REQUIRES_NEW,ISOLATION_DEFAULT (Session #2009738365)
2018-03-02 15:45:41,637 INFO [Caesium-1-4] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers synchronised [ 27453 ] users in [ 3598ms ]
2018-03-02 15:45:41,637 INFO [Caesium-1-4] [atlassian.crowd.directory.DbCachingRemoteDirectory] synchroniseCache failed synchronisation complete for directory [ 220037125 ]in [ 50918ms ]
2018-03-02 15:45:41,650 ERROR [Caesium-1-4] [atlassian.crowd.directory.DbCachingDirectoryPoller] pollChanges Error occurred while refreshing the cache for directory [ 220037125 ].
org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy61.commit(Unknown Source)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.atlassian.crowd.directory.$Proxy2592.updateUsers(Unknown Source)
at com.atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations.addOrUpdateCachedUsers(DirectoryCacheImplUsingChangeOperations.java:65)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllUsers(UsnChangedCacheRefresher.java:190)
at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseAll(AbstractCacheRefresher.java:56)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAll(UsnChangedCacheRefresher.java:163)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1186)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:74)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:50)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:96)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.doRunJob(JobRunnerWrapper.java:120)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.lambda$runJob$0(JobRunnerWrapper.java:91)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.runJob(JobRunnerWrapper.java:91)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeLocalJob(CaesiumSchedulerService.java:410)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:388)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34)
at java.lang.Thread.run(Thread.java:745)
I tried to flush some caches (Embedded Crowd Users, Embedded Crowd Directory), to change username and lower_username when I saw they were differents in the user_mapping table with no effective result.
Does anyone got this issue before ?
Thanks.