Crowd: Error when synchronising with Active Directory

[LEM]

The sync doesn't work anymore. Error in the Logs as follows. Any ideas what this means, or how I can resolve this?

2019-03-20 09:07:03,620 Caesium-2-1 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: ORA-01502: index 'CROWD.SYS_C00100750' or partition of such i
ndex is in unusable state
], SQL: insert into cwd_synchronisation_status (directory_id, node_id, sync_start, sync_end, sync_status, status_parameters, id) values (?, ?, ?, ?, ?, ?, ?)
2019-03-20 09:07:03,622 Caesium-2-1 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 1502, SQLState: 72000
2019-03-20 09:07:03,622 Caesium-2-1 ERROR [engine.jdbc.spi.SqlExceptionHelper] ORA-01502: index 'CROWD.SYS_C00100750' or partition of such index is in unusable state

2019-03-20 09:07:03,622 Caesium-2-1 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 1502, SQLState: 72000
2019-03-20 09:07:03,622 Caesium-2-1 ERROR [engine.jdbc.spi.SqlExceptionHelper] ORA-01502: index 'CROWD.SYS_C00100750' or partition of such index is in unusable state

2019-03-20 09:07:03,622 Caesium-2-1 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.GenericJDBCException: could not execute batch]
2019-03-20 09:07:03,622 Caesium-2-1 ERROR [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 32770 ].
org.springframework.orm.hibernate5.HibernateJdbcException: JDBC exception on Hibernate data access: SQLException for SQL [insert into cwd_synchronisation_status (directory_id, node_id, sync_start, sync_end, s
ync_status, status_parameters, id) values (?, ?, ?, ?, ?, ?, ?)]; SQL state [72000]; error code [1502]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not exe
cute batch
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:248)
at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:755)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:594)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:765)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:734)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:518)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:292)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy64.syncStarted(Unknown Source)
at com.atlassian.crowd.manager.directory.SynchronisationStatusManagerImpl.syncStarted(SynchronisationStatusManagerImpl.java:47)
at sun.reflect.GeneratedMethodAccessor917.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy65.syncStarted(Unknown Source)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:76)
at sun.reflect.GeneratedMethodAccessor916.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy71.synchronise(Unknown Source)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:85)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)

[sgorgi · Accepted solution]

Hi Werner,

We faced the same issue after upgrading Crowd from version 3.1.3 to version 3.4.5 on RHEL connected to a Oracle database.

You have to rebuild the index in your database for each directory that is affected.

1. Simply tail your crowd log and try to synchronize your directory using the GUI
2. You will find something like:

   ORA-01502: index '<DIRECTORY-INDEX>' or partition of such index is in unusable state

3. Connect to your database, e.g. with Oracle SQL Developer and search for the index <DIRECTORY-INDEX>
4. You will see, that STATUS has the value UNUSABLE. Rebuild your index.
5. STATUS should now be switched to VALID and synchronization of your directory should work again

Hope this is also working for you.

All the best,

Sascha

[Werner Lembacher]

-

[LEM]

Thanks Sascha, this worked for us!