Issue with Active Directory Synchornization

[Aaron Williams]

Issue with AD synchronization.

2017-08-10 16:52:19,892 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteDirectory] Incremental synchronisation for directory [ 10100 ] was not completed, falling back to a full synchronisation
bCachingRemoteDirectory] Incremental synchronisation for directory [ 10100 ] was not completed, falling back to a full synchronisation
2017-08-10 16:52:19,892 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteDirectory] INCREMENTAL synchronisation for directory [ 10100 ] was not successful, attempting FULL
2017-08-10 16:52:21,092 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 1046ms
2017-08-10 16:52:22,632 CrowdUsnChangedCacheRefresher:thread-1 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Domain User Accounts,DC=XXXX,DC=ORG took 1034ms
2017-08-10 16:52:26,082 CrowdUsnChangedCacheRefresher:thread-1 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Domain User Accounts,DC=XXXX,DC=ORG took 1253ms
2017-08-10 16:52:27,033 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 5939ms
2017-08-10 16:52:28,928 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 1894ms
2017-08-10 16:52:30,176 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 1247ms
2017-08-10 16:52:32,506 CrowdUsnChangedCacheRefresher:thread-1 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Domain User Accounts,DC=XXXX,DC=ORG took 1323ms
2017-08-10 16:52:32,573 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 2397ms
2017-08-10 16:52:34,377 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 1803ms
2017-08-10 16:52:35,392 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for search with handler on OU=Groups,DC=XXXX,DC=ORG took 1013ms
2017-08-10 16:52:35,738 CrowdUsnChangedCacheRefresher:thread-1 INFO ServiceRunner [c.a.c.d.ldap.cache.UsnChangedCacheRefresher] found [ 17438 ] remote users in [ 15846ms ]
2017-08-10 16:52:36,318 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteChangeOperations] scanned and compared [ 17438 ] users for delete in DB cache in [ 556ms ]
2017-08-10 16:52:36,318 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteChangeOperations] scanned for deleted users in [ 556ms ]
2017-08-10 16:52:36,813 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteChangeOperations] scanning [ 17438 ] users to add or update
2017-08-10 16:52:36,850 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DirectoryCacheImplUsingChangeOperations] scanned and compared [ 17438 ] users for update in DB cache in [ 532ms ]
2017-08-10 16:52:36,851 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteChangeOperations] updating [ 49 ] users
2017-08-10 16:52:38,650 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteChangeOperations] updated [ 49 ] users successfully in [ 1799ms ]
2017-08-10 16:52:38,650 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DirectoryCacheImplUsingChangeOperations] synchronised [ 17438 ] users in [ 2332ms ]
2017-08-10 16:52:42,396 CrowdUsnChangedCacheRefresher:thread-2 INFO ServiceRunner [c.a.c.directory.ldap.SpringLdapTemplateWrapper] Timed call for lookup on cn=scriptlogictest,ou=security,ou=groups,dc=XXXX,dc=org took 7002ms
2017-08-10 16:52:42,397 Caesium-1-2 INFO ServiceRunner [c.a.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 10100 ] in [ 22505ms ]
2017-08-10 16:52:42,472 Caesium-1-2 ERROR ServiceRunner [c.atlassian.scheduler.JobRunnerResponse] Unable to synchronise directory
com.atlassian.crowd.exception.OperationFailedException: java.util.concurrent.ExecutionException: org.springframework.ldap.CommunicationException: adldap.XXXX.org:636; nested exception is javax.naming.CommunicationException: adldap.XXXX.org:636 [Root exception is java.net.NoRouteToHostException: No route to host]
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllGroups(UsnChangedCacheRefresher.java:237)
at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseAll(AbstractCacheRefresher.java:91)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAll(UsnChangedCacheRefresher.java:168)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1122)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:76)
at com.atlassian.jira.crowd.embedded.JiraDirectorySynchroniser.synchronizeDirectory(JiraDirectorySynchroniser.java:77)
at com.atlassian.jira.crowd.embedded.JiraDirectorySynchroniser.runJob(JiraDirectorySynchroniser.java:52)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:453)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:447)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:394)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:279)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:275)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:78)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:70)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:40)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.springframework.ldap.CommunicationException: adldap.XXXX.org:636; nested exception is javax.naming.CommunicationException: adldap.XXXX.org:636 [Root exception is java.net.NoRouteToHostException: No route to host]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllGroups(UsnChangedCacheRefresher.java:218)

[somethingblue]

Hi Aaron,

One thing that stands out is the following:

com.atlassian.crowd.exception.OperationFailedException: java.util.concurrent.ExecutionException: org.springframework.ldap.CommunicationException: adldap.XXXX.org:636; nested exception is javax.naming.CommunicationException: adldap.XXXX.org:636 [Root exception is java.net.NoRouteToHostException: No route to host]

Take a look at the java.net.NoRouteToHostException in LDAP connection knowledge base article for more info on how to diagnose the issue:

JIRA is not able to connect to the LDAP server because it is not able to resolve the path that it can use to the LDAP/Active Directory server.

There can be several reasons for this : 

• There might be a reverse proxy in place that is blocking the connection
• There might be a firewall in place between the servers that is blocking the port
• JIRA server not being able to resolve the hostname of the LDAP server correctly, etc
• There may be HTTP Access policies being pushed from ActiveDirectory, or puppet (preventing access to the LDAP Server)

Having said this, this is a network issue instead of a JIRA issue.

Cheers,

Branden

[Steven F Behnke]

Lol...

No detail? Not even a code block for the error text? Has this ever worked before? Did this just break? Have you changed anything?

Consider reading the KB articles on the error: https://confluence.atlassian.com/jirakb/java-net-noroutetohostexception-in-ldap-connection-703398693.html