nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Why does our JIRA's open file descriptor count plateau at 3500?

unknown

After upgrading from JIRA 4.4.4 to 5.2.4, we noticed that the number of JIRA's open file descriptors started reaching higher numbers than we'd seen before, eventually exceeding the ulimit, which caused various problems. We've since adjusted the ulimit and are OK, but we noticed last week that the number of open descriptors climbed to about 3500, and then plateaued there for several days. Many/most of these point to index cache files that had been deleted.

We don't know if this is normal or a symptom of a problem. Why 3500? Why deleted files? Can we safely assume that JIRA or java will manage these descriptors in a way that won't exceed our ulimit settings (currently 8192)? Are there administrative practices we need to perform to avoid a problem with this?

Thanks.

Tom

Accepted answers

unknown

We don't have all the details resolved, and certainly, we never again saw this "flat top" in the charts for the open file descriptions, but I'm not sure that matters. The long standing problem we experienced with unbounded growth of open file descriptors was the more serious problem.

The file descriptor leak stopped when we moved the cache directories to a local/native file system rather than an NFS mounted one. We're still digging to see exactly why we can't run the way we did before the upgrade, and it may turn out to be some interaction between specific versions of the OS, NFS, Java, and of course JIRA.

For performance reasons, we may never go back to the NFS hosted cache directories, but moving away from them certainly solved our problem.

All comments

Renjith Pillai

Do you have some actual numbers? Will be interesting to check whether is an actual bug.

unknown

This is what we see in Java Melody for a week that includes the phenomenum

unknown

During the "flat top" period, about 80% of the descriptors referred to deleted files. The plateau ended when we rebooted the system as part of routine maintenance.

Renjith Pillai

I am guessing that this is being worked upon by Atlassian support. Let us wait for the results.

unknown

Yes - Our local team has an issue actively being worked (JSP-149435) and presumbly that will yield something in the near term.

unknown

FYI - This is still being worked, with lots of ideas, but still no rock solid understanding of the underlying problem, nor a real workaround except to set the ulimit very high and restart the system periodically.

unknown

Nic Brough -Adaptavist-

Out of interest, when you say "NFS hosted cache directories" - do you mean you had the Jira "cache" directories on an NFS drive?

unknown

Yes. The entire install (product and data directories) have been on NFS file systems since we first started using JIRA, because we run JIRA on muliple hosts but kept resources for all the instances on the same file system, which included the cache directories. Now the <JIRADATA>/caches directory is a sym link to a directory in /var, and the leak has stopped.

JamieA

No help to you but I have seen similar problems hosting Nexus's lucene index on an NFS-mounted dir. Long and short of it is, keep Lucene well away from NFS.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

jira

descriptor