Confluence servers appear go down once a day for up to 10 seconds

[Matthew Maule]

Hello,

This is more of a curiosity than anything since it doesn't actually affect usability. We have 2 self-hosted Confluence servers of version 7.6.2, both running on Windows server 2019 v1809. Both servers are proxied through a Kemp load balancer for https. We have used several versions now on server 2019 and never had any issues whatsoever with Confluence applications.

After a recent round of Windows updates we started getting a series of alerts each day, for both servers, from the Kemp load balancer that the servers were going down and then immediately coming back up. They are only gone for about 3-4 seconds usually, but sometimes as much as 10 seconds. This has not affected usability that we can tell as there have been no user reports of the sites not loading, and it usually happens around 1AM local time so nobody should be online anyway.

I have checked in Event Viewer in Windows on both machines and neither has anything that lines up with the times of the alerts. I also checked that it wasn't happening during the automated database backups, it is off by about an hour from when the backups run. There are 15 other web servers proxied through the same Kemp load balancer that all received the same Windows updates around the same time that do not get these alerts. It is only on the Confluence servers and they started at the same time.

I have never dug into any of the server logs before so I wanted to see if anyone has any ideas on what this could be or where I should look for more information/what I should be looking for.

Thanks!

[Diego]

Hello @Matthew Maule !

That is an interesting behavior you have at hand! Since you already looked into Windows logs, I believe that we will need to look into Confluence logs.

You can find the logs inside this folder:

<confluence-home>/logs/

Here is more information about the home folder:

• [Confluence Server] Confluence Home and other important directories

It is quite difficult to know, right off the bat, what is happening with your instance. There are many variables at play. Not looking at the logs makes it even harder.

Taking in consideration that this happens at 1 AM, my hunch would be some scheduled job running and making a slight hiccup in your server. A job that comes to mind is the automatic backup. The only inconsistency here is that it is scheduled to run at 2 AM by default.

You can check more about scheduled jobs and backups here:

• [Confluence Server] Scheduled Jobs
• [Confluence Server] Configuring Backups

So, my tips would be to start with the following:

1. Check Confluence logs
2. Check what happens before, during and right after the downtime occurrence
3. Check memory consumption during the downtime window
4. Check for scheduled jobs

Let us hear from you!