LDAP Connection Sync issues - Users can't login often

[SaSu]

Hello,

we have a Jira Server Instance with a connected Microsoft Active Directory user directory. 

Since we upgraded to Jira 8.* we have massive problems with the LDAP sync.

Very often users are not able to login. Most often the automatic sync times out. 

When users complain about not being able to login we try to trigger the sync manually. Sometimes it works then. 

Maybe someone has an idea of what to do?

We found this error message in our logs:

---

atlassian-jira.log

Caesium-1-2 ERROR ServiceRunner [c.a.crowd.directory.DbCachingRemoteDirectory] Incremental synchronisation for directory [ 10000 ] was unexpectedly interrupted, falling back to a full synchronisation

org.springframework.ldap.UncategorizedLdapException: Uncategorized exception occured during LDAP processing; nested exception is javax.naming.NamingException: LDAP response read timed out, timeout used: 600000 ms.; remaining name '/'

at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:228)

at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:397)

at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:440)

at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper$2.timedGet(SpringLdapTemplateWrapper.java:128)

at com.atlassian.crowd.directory.ldap.monitoring.TimedSupplier.get(TimedSupplier.java:37)

at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper.invokeWithContextClassLoader(SpringLdapTemplateWrapper.java:85)

at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper.lookup(SpringLdapTemplateWrapper.java:117)

at com.atlassian.crowd.directory.MicrosoftActiveDirectory.fetchInvocationId(MicrosoftActiveDirectory.java:714)

at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.synchroniseChanges(UsnChangedCacheRefresher.java:113)

at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1080)

at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:82)

at com.atlassian.crowd.audit.NoOpAuditLogContext.withAuditLogSource(NoOpAuditLogContext.java:17)

at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:80)

at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:48)

at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:92)

at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)

at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)

at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)

at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)

at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:430)

at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:454)

at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:382)

at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)

at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)

at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)

at java.base/java.lang.Thread.run(Unknown Source)

Caused by: javax.naming.NamingException: LDAP response read timed out, timeout used: 600000 ms.; remaining name '/'

at java.naming/com.sun.jndi.ldap.LdapRequest.getReplyBer(Unknown Source)

at java.naming/com.sun.jndi.ldap.Connection.readReply(Unknown Source)

at java.naming/com.sun.jndi.ldap.LdapClient.getSearchReply(Unknown Source)

at java.naming/com.sun.jndi.ldap.LdapClient.search(Unknown Source)

at java.naming/com.sun.jndi.ldap.LdapCtx.doSearch(Unknown Source)

at java.naming/com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)

at java.naming/com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(Unknown Source)

at java.naming/com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(Unknown Source)

at java.naming/javax.naming.directory.InitialDirContext.search(Unknown Source)

at java.base/jdk.internal.reflect.GeneratedMethodAccessor746.invoke(Unknown Source)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.base/java.lang.reflect.Method.invoke(Unknown Source)

at org.springframework.ldap.transaction.compensating.manager.TransactionAwareDirContextInvocationHandler.invoke(TransactionAwareDirContextInvocationHandler.java:90)

at com.sun.proxy.$Proxy4116.search(Unknown Source)

at com.atlassian.crowd.directory.ldap.SpringLdapTemplateWrapper$2.lambda$timedGet$0(SpringLdapTemplateWrapper.java:124)

at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:363)

... 24 more

---

catalina.out

WARNING [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread [http-nio-8090-exec-11 url: /plugins/servlet/embedded-crowd/directories/troubleshoot/; user: xxx.xxx] (id=[29]) has been active for [127,419] milliseconds (since [6/11/22 1:58 PM]) to serve the same request for [xxx/plugins/servlet/embedded-crowd/directories/troubleshoot/] and may be stuck (configured threshold for this StuckThreadDetectionValve is [120] seconds). There is/are [1] thread(s) in total that are monitored by this Valve and may be stuck.

java.lang.Throwable

at java.base@11.0.13/jdk.internal.misc.Unsafe.park(Native Method)

at java.base@11.0.13/java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

at java.base@11.0.13/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)

at java.base@11.0.13/java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapRequest.getReplyBer(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.Connection.readReply(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapClient.getSearchReply(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapClient.search(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapCtx.doSearch(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapCtx.searchAux(Unknown Source)

at java.naming@11.0.13/com.sun.jndi.ldap.LdapCtx.c_search(Unknown Source)

[Daniel Eads {unmonitored account}]

Hi Sandra,

There are a couple ways of doing this. From the web interface, the quickest is to find the user in the User browser from Administration and select View Project Roles from the action menu:

This will give you a matrix of all the projects and what roles the user is associated with.

There are of course a couple problems looking at users like this:

1. If you have a lot of roles and/or projects, that page is pretty beefy.
2. If a previous administrator didn't do their job well and added users directly to schemes instead of using Roles, you won't see what projects that user is on from this screen (for the direct grants).

So, if you're on Server and have access to SQL, you can take this a step further by directly querying your database for any schemes that user has been added to.

Replace the 'username' (at the end of the query) with the username you're looking to find in the following queries.

Notification scheme adds:

select notif_type, notif_parameter, name from notification
join notificationscheme on notification.scheme = notificationscheme.ID
where lower(notif_parameter) = lower('username')

Permission scheme adds:

select perm_parameter, permission_key, name from schemepermissions
join permissionscheme on schemepermissions.scheme = permissionscheme.id
where lower(perm_parameter) = lower('username')

If you by chance had a predecessor who added groups to notification schemes, you can also expand this even a little further to find where notifications might be coming from based on group membership to notification schemes. Note that there are two places in this query where you'd need to drop in the username.

Notification scheme adds, including notification through group memberships:

select notif_type, notif_parameter, name from notification
join notificationscheme on notification.scheme = notificationscheme.ID
where lower(notif_parameter) in (select lower_parent_name from CWD_membership where lower_child_name = 'username')
or lower(notif_parameter) = lower('username')
order by notif_parameter

Hope that helps!

[Audra]

Hey Sandra,

If you go to Issues > Search for issues, and click Advanced next to the magnifying glass, you can sort issues by a number of criteria (including user).