Unable to authenticate users recently added to LDAP

[Harrison Barnett]

There has been an anomaly noted the last few days where users recently added to active directory are not able to authenticate through the LDAP user directories configured in our Jira instance. For context, we upgraded our LDAP user directories to use SSL last October and have noted no issues. For further context, our user directories were initially configured with two user directories per zone with one (I assume) designated for authentication and the other set to synchronize at an interval to (again, I assume) keep whatever caches up-to-date with any LDAP changes.

The user directory order goes like:
Zone 1
Zone 2
...
Zone N
Zone 1(sync)
Zone 2(sync)
...
Zone N(sync)

I disabled the directories that sync nearly two weeks ago in order to cut down on the volume of requests to the LDAP server with no interruption noted with users logging in.

However, yesterday a robot account was created in our AD for use in some automations, and we attempted to add this new account to Jira. When trying to login with this account, I get the message "Sorry, your username and password are incorrect - please try again." I read that I may want to change the password of this account in active directory and try again, which I did, with no change in this behavior. The user has application access via group permissions. This user can be logged into other systems using its AD credentials.

The `atlassian-jira-security.log` displays these errors:

2024-01-10 09:06:35,313 https-jsse-nio-443-exec-25 anonymous 546x145341x3 bxkryf <some.ip.address> /login.jsp login : 'username' tried to login but they do not have USE permission or weren't found. Deleting remember me cookie.

2024-01-10 09:06:35,313 https-jsse-nio-443-exec-25 anonymous 546x145341x3 bxkryf <some.ip.address> /login.jsp The user 'username' has FAILED authentication. Failure count equals 5

2024-01-10 09:06:35,313 https-jsse-nio-443-exec-25 anonymous 546x145341x3 bxkryf <some.ip.address> /login.jsp login.jsp called with lastLoginResult : com.atlassian.jira.bc.security.login.LoginResultImpl@1111e315[reason=AUTHENTICATED_FAILED,loginInfo=com.atlassian.jira.bc.security.login.LoginInfoImpl@42277812[lastLoginTime=<null>,previousLoginTime=<null>,loginCount=<null>,currentFailedLoginCount=5,totalFailedLoginCount=5,lastFailedLoginTime=1704895595313,elevatedSecurityCheckRequired=false,maxAuthenticationAttemptsAllowed=9223372036854775807],userName=username,deniedReasons=[]]

I went and looked at the user directory configuration for the zone that I *know* this user exists on a ran a quick test against my credentials:

And then I ran the same quick test against the newly added user:

 

No error messages or anything, just flatly telling me that the user doesn't exist in the AD server where I *know* both of our accounts live. What could possibly account for this behavior? 

[Harrison Barnett · Accepted solution]

After lots of phone calls and digging through Splunk logs, I found out that the robot acct that was giving me so much trouble belonged to an organizational unit that our user directories were not configured to search for. Apparently when users are created on the backend for us, robot accounts like the one I requested are sometimes added to a separate organizational unit from human users, so Jira never had a chance of finding it. I appreciate everyone's input as it has helped me understand a little better how user directories are meant to be utilized. 

Cheers!

[Steinar Line - Kantega SSO]

Hi guys, I have read through your discussion. First of all, I would expect it to be not a common setup to have two directory configurations for each actual directory. Also, authentication should be possible against the synced directories, so you should not need the delegated ones.

You mention that LDAP traffic was high, but I would expect the cost of the sync to be quite low since only changes are synced. You may also modify in user directory configuration to "Update group memberships when logging in" - "For newly added users only" to make LDAP traffic lower where group memberships only would update on each sync interval (by default every 60 mins).

If you want to use the delegated directories instead of the synced ones, I believe you would have to configure to "Copy the user on login" on the delegated directory and also tick off some more of the checkboxes in the below screenshot:

If not, I belive you will see something like this in the atlassian-jira-security.log as I did when I tested this now:

/login.jsp login : 'username' tried to login but they do not have USE permission or weren't found.

So to sum up, I suggest to either only use the synced directories and then to lower LDAP traffic set group checkbox to "For newly added users only". We recommend this to our SSO customers when having traffic issues. Or you could enable to copy user on login and also synchronise group memberships for the delegated directories.

I would expect that these options should work and it seems unneccesary to define two directory configs like you currently have. I would also expect that the delegated ones would not be used if they were prioritised below the synces ones on the User Diretory page.

Do say if you find a good setup for your directories.

Best regards
Steinar
Kantega SSO

[Harrison Barnett]

Steinar, 

Thanks for your response! I had a feeling that it was unnecessary to maintain redundant directories like this, and I have plans to consolidate all of our directories into a single source for authentication. Tangential to my original post, all of the LDAP zones that we have configured share the same LDAP root, so I'm wondering how possible it would be to "migrate" users from Zone 1, Zone 2 ... Zone N to Zone Consolidated. I put "migrate" in quotes because I'm not unconvinced that this would be difficult since Jira doesn't like to migrate users from one directory to another if they exist in both. I'd love to be able to do this while preserving all of the current information about users, such as the groups they belong to, the projects they belong to, last login date, etc. 

As for the original topic of my post, after some digging around I found out that the powers that be will sometimes put robot accounts like the one that was giving me so much trouble in their own organizational unit, so Jira couldn't find this user at all since it's not searching LDAP at that ou. So this issue is more or less resolved as it's just going to take some config on my end to make Jira search for this class of LDAP user.

I appreciate your response though! And please do let me know if you have any input regarding my other question regarding consolidating directories. 

Regards,
Harrison

[Steinar Line - Kantega SSO]

You are right that a user with a certain username moving from one directory in Jira to another will lose his group relations, while the user's actions in Jira (like his created issues, comments and such) are linked to the username and will be kept. So moving users to different directories may be a cumbersome process as you may need to re-establish some group relations giving access to projects and so.

In our product Kantega Single Sign-on we do have a Copy directory feature to copy  users (if you select to copy these), and also groups (if they do not already exist) and memberships which links users in the "To directory" to the same groups in the "From directory". This may be useful in certain of such migration operations. AD can also be the "To directory" as long as this is set up as read/write.

Steinar

[Elias Brattli Sørensen - Kantega SSO]

Hi Harrison,

Welcome to the community! I might be misunderstanding your setup, but I have a few thoughts on the topic. Won't disabling the directories that sync from Zone 1 potentially cause problems like this, because users added to Zone 1 are no longer known to and searchable in Jira because of outdated caches? The users added the last two weeks will no longer get copied to the user cache in Jira user database, and maybe LDAP authentication actually doesn't work properly without replicating the users with the sync.

Also, are the user and group schema settings etc. equivalent for "Zone 1 sync" & Zone 1 "non-sync"?  It might be that only the users that previously have been synced through Zone 1 sync (users in the cache) are now known to Jira with your setup.

Have you considered enabling syncing from the "authentication" LDAP directory? This sounds better that your old approach of having what seems like two copies / references to the same LDAP, but also probably better suited for integrating the systems than not syncing at all.

Best regards,
Elias
Kantega SSO
www.kantega-sso.com

[Harrison Barnett]

Hi Elias, thanks for the response. I actually did exactly that, reenabled the other directories and allowed them to synchronize with the same, exact behavior noted.

I guess my other question would be this: why would someone set up user directories this way in the first place? Why not just set up a delegated LDAP user directory for each zone for authentication, have folks added to the system automatically on first login and leave it at that? The way that it used to work is that we'd create a user with the uid as their user name as defined in AD, have them login, the login would fail, manually grant them access by adding them to a group with software access and call it a day. It's unclear to me why it would require TWO user directories to do this. 

If it's unclear, I was left zero documentation on why it was configured this way, and it really doesn't make much sense to me. It seems like a single user directory talking to LDAP should be able to look up whoever I need it do if they exist in it. 

**NB: I edited the original post as some of it got caught in the code formatting block**