Users are becoming inactive in Jira Service Desk

[Dennis Shushunov]

Hi

Our Jira SD is configured to sync with an LDAP repository. Users from the repository from time to time become "Inactive". After manual re-sync everything gets back to normal.

Please advise.

[Andy Heinzer]

Hi Dennis,

I was taking a look through the other support case you have for this issue and I found a number of log entries like so:

2019-08-30 22:52:06,731 Caesium-1-2 WARN ServiceRunner     [c.a.c.d.ldap.mapper.UserContextMapper] Failed to map attribute <gidNumber> from context with DN <uni=abc1234,ou=People,o=XYZ University,c=US>
java.lang.NullPointerException 

It's not clear to me if this is a factor in the problem with syncing failing occasionally, but it is so filling the logs that I think it would be best to address that issue first and then see if we can gather updated logs about this sync problem.  I found related KB to help with that log message Directory synchronization throws "Failed to map attribute <gidNumber>"  From that KB there are steps you can take to avoid or hide that error in the logs:

Resolution

This WARN message is usually harmless, but there are two ways to solve this issue:

• Make sure that the LDAP user being referred in the WARN message has the gidNumber attribute;

• Or, you can use a LDAP filter, as the following example, to only return LDAP users that have the gidNumber attribute.

  (&(objectclass=inetorgperson)(gidnumber=*))

Perhaps following the 2nd resolution step here might help avoid the source problem, but even if it does not, then by eliminate this kind of error message from the logs will help us to better see other errors/warning in the logs when Jira does attempt to sync.

Also, going forward, since we know you have a support case for this issue, it would probably be best to work on this issue there in the other support portal.  We would prefer to not duplicate troubleshooting efforts here if possible.  Then once we have a solution on that support case, it would be helpful to Community to update this thread with the steps needed to correct this scenario. That way other users that might have the same problem could potentially find the same solution.

Cheers,

Andy

[Dennis Shushunov]

Andy

Thank you for looking into it. Unfortunately neither solution for resolving gidNumber issue would work for us, because all our entries do not have this attribute.

I am surprised that Jira is trying to map this attribute given that we have selected to sync users only, without group membership.

We will try to resolve this in the support case and if I will update this thread if it will be resolved. I've asked this question here when we did not yet have support contract.

[Andy Heinzer]

Hi Dennis,

I understand that you have Jira connected to an external LDAP user directory, and that sometimes these users are marked as inactive.   We have seen this problem in Jira before in support.  It tends to happen in environments that have a very large number of users/groups to sync over.

To troubleshoot this issue, I would recommend a two-pronged approach to start:

1. Reduce the number of groups/users to sync over to Jira.  This can be done though the use of LDAP filters.  I would recommend our document in Reducing the number of users synchronized from LDAP to JIRA applications.  Many times we find that Jira can be connected to an LDAP site without any filtering, which can bring into Jira many more users than are actually ever going to try to login to Jira.   Filtering these can remove those unused accounts, and reduce the overhead on Jira itself
2. You can also adjust the frequency in which Jira syncs to that directory.  Kelly Wong has a great answer on how to do this in https://community.atlassian.com/t5/Jira-questions/Re-where-to-adjust-the-sync-setting-for-an-AD-group/qaq-p/930613/comment-id/298549#M298549 By default Jira sets to this sync ever 60 minutes.  If you find you can't reduce the number of users to sync to Jira, then perhaps it could help to increase this interval to a value such as 300 or 600.  The trade off here is that the longer this interval becomes, the more time it takes for Jira to see new users in the LDAP directory.

Aside from that approach, we would want to take a closer look at the logs when this happens, say in the $JIRAHOME/log/atlassian-jira.log file.  This should have records of the sync process, such as start time, completion, and possibly number of records to sync over.  Perhaps with more information we can better advise you on this topic.

Regards,

Andy

[Dennis Shushunov]

Hi Andy

Many thanks for your response.

1. We only synchronize about 1000 users from LDAP. Don't sync any groups. Is this considered "very large"? According to Jira, the sync itself takes only about 4 seconds.

2. We sync once every 10 hours (600 min), so the interval is much more than the default.

All inactive users become active again after manual synchronization, however auto-sync does not make them active.

Best regards

Dennis 

[Andy Heinzer]

No, 1000 users is not really considered large.  And taking 4 seconds to sync seems reasonable given that size.  I'm surprised to hear that you might be encountering these kinds of problems syncing ~1000 users and no groups.   I take it that you must be using read only with local groups.  This way you can at least grant these users group membership needed in Jira to then grant application access.   This is helpful to know because we at least know we are not using the LDAP groups here to sync.  Maybe there is something else happening in your configuration here.

1. What version of Jira is this?
2. Why set the sync interval so high?  

In your case, I would actually think that the opposite trend for sync interval could be beneficial here.   If a sync fails for an unknown reason, what suspect is happening is that those users are getting removed but failed to be added back because the sync fails for a yet unknown reason.  In turn, users could have to wait up to 10 hours before another sync might be attempted which would restore them.  Given the size of your userbase, it seems like resetting the interval back to 1 hour (60 minutes) might be better here.    Unless this sync is putting a high overhead/strain on the LDAP server, it feels like this might help avoid the problem in general.  

While we might be able to learn more about this problem if we can look at the $JIRAHOME/log/atlassian-jira.log file at a time when this sync fails, an alternative approach could be to move away from using the syncing LDAP directory type altogether, and instead create a delegated LDAP directory within Jira.  This does not sync users, but instead when a user attempts to login, it looks up that user account in LDAP and adds them to Jira if they provide the correct credentials.  More details in What is the Difference between Connector and Delegated LDAP User Directories?.

[Dennis Shushunov]

Thank you for your response.

1. This is Jira Service Desk 4.2.2
2. The interval is set so at the request of our identity group administrators.

We are using "read-only with local groups mode".

I am not sure that changing interval will help in this case, mostly because i happened to check on it a few minutes after auto sync was performed (according to the "Last synchronized..." message) and there still were some inactive users. Also the same message is always says that the sync completed successfully.

Would you help me to find answer to the following questions:

• Why is there a mechanism to make users inactive? What purpose it supposed to serve?
• How manual and auto syncs are different? - Why would manual sync fix the issue but not the auto?

We cannot use delegated LDAP because we would like to be able to use users who have not (yet) logged in to Jira SD (for approvals, assignments etc).

I have changed some of the timeouts yesterday and so far no users were marked inactive. Maybe it has fixed the issue.

Thank you very much.

[Andy Heinzer]

Hi Dennis,
I will try to answer your questions here as best I can:

Why is there a mechanism to make users inactive? What purpose it supposed to serve?

If Jira fails to complete a sync to the LDAP server, it does not know what users actually exist there.  Since Jira is then actually dependent on being able to contact that user server (LDAP) to authenticate these accounts, it needs to be able to see which users exist there.   Another reason this mechanism exists is to make sure that users don't remain active in Jira, when that user has been disabled/deleted from LDAP.   It certainly appears that a failed sync is behaving much like a set of users that are simply not found in LDAP.

How manual and auto syncs are different? - Why would manual sync fix the issue but not the auto?

Auto syncs are being run on a schedule within Jira.  This happens based on the settings in that user directory configuration in Jira.  This sync is being run by the scheduled services in Jira.  But a manual sync is called by a Jira admin, it is also not run on a set interval, but instead it runs right now.   I think one of the major differences in the way these two are different in your environment could be just the time of day each sync runs.  

I could see a scenario when say the LDAP server becomes unresponsive say for a nightly maintenance, Jira's automatic sync process could fail when the server becomes unresponsive here.  If that happens, with a large sync interval like yours of 10 hours, a single sync failure means users would be unable to login for up to 10 hours.  It's possible that the next sync will probably succeed to correct this, but it's more likely that an admin will manually sync it to correct it before that time.

Extending the timeouts seems like a good next step here.  Please let me know if you run into any more trouble here.

Andy

[Dennis Shushunov]

Thank you very much for your response.

Unfortunately nothing seem to help here. The sync period was reduced from 600 to 120 minutes and all timeouts are 0. 

The users still became inactive.