We're having an issue with our locally hosted implementation of Confluence. The last three weekends on Saturday afternoon the catalina.out starts getting blasted with the following errors:
11-Apr-2020 20:27:21.247 SEVERE [https-jsse-nio-8443-Acceptor] org.apache.tomcat.util.net.Acceptor.run Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:461)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:73)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)
at java.lang.Thread.run(Thread.java:748)
java.util.logging.ErrorManager: 0: Unable to delete log files older than [90] days
11-Apr-2020 20:27:21.298 SEVERE [https-jsse-nio-8443-Acceptor] org.apache.tomcat.util.net.Acceptor.run Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:461)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:73)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)
at java.lang.Thread.run(Thread.java:748)
We had a scheduled task for application backups enabled, I saw the recommendation was to disable that which I did since we run nightly AWS snapshots on the application server as well as the database. Disabling these backups did not resolve the issue, when this error occurs the front end becomes completely unresponsive and times out. Once the application is bounced it comes right back and is good for another week.
Confluence specific content of /etc/limits.conf:
# End of file
confluence soft nofile 32768
confluence hard nofile 32768
