Why was my Confluence-instance down?

[Florian]

This morning my Confluence was not running. I have it set up as a service on Ubuntu 16.04. I had to manually start it

$ sudo /etc/init.d/confluence start

I wanted to attach the catalina.out-log to my post, however this message cannot exceed 20000 characters, so in what log file should I look for errors?

[Florian · Accepted solution]

thanks for that hint, I found the reason. My backup-routines (run by CRON-jobs) shut down both, confluence AND PostgreSQL due to backups and the time between shutting them down was too short that in some cases (like it happened yesterday) the database was unavailable before confluence was shut down. I will fix it by better timing of the CRON-Jobs

[Shannon Spaniol]

Ooh! Good catch. :) Glad I got you pointed in the right direction at least!

Take care and Kind Regards,
Shannon

[Shannon Spaniol]

Hi there,

You can try the atlassian-confluence.log file and it might explain the reason for the crash there. 

A few examples of things that might cause a crash are available here:

• Troubleshooting Confluence Hanging or Crashing

Let us know if you find any suspicious lines around the time of the crash and if you're not able to determine the cause please feel free to share with us.

Kind Regards,
Shannon

[Florian]

In atlassian-confluence.log I have a lot ERROR-entries like those the evening before that happened:

09-24 22:00:04,911 ERROR [localhost-startStop-5] [engine.jdbc.spi.SqlExceptionHelper] logExceptions FATAL: terminating connection due to administrator command
2017-09-24 22:00:04,911 ERROR [localhost-startStop-5] [engine.jdbc.spi.SqlExceptionHelper] logExceptions An I/O error occurred while sending to the backend.
2017-09-24 22:00:04,912 WARN [localhost-startStop-5] [impl.schedule.caesium.SchedulerClusteredJobDaoProxyWrapper] delete Could not delete clustered job 'DefaultPluginScheduler:com.atlassian.analytics.client.upload.S3EventUploader:job'
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is net.sf.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection

2017-09-24 22:00:04,914 ERROR [localhost-startStop-5] [engine.jdbc.spi.SqlExceptionHelper] logExceptions FATAL: terminating connection due to administrator command
2017-09-24 22:00:04,914 ERROR [localhost-startStop-5] [engine.jdbc.spi.SqlExceptionHelper] logExceptions An I/O error occurred while sending to the backend.

followed by some hundred similar entries

[Shannon Spaniol]

Hello,

It appears that your database connection is the root cause Confluence crashing.

Have a look at your database logs to see why it might be closing the connection to Confluence. You may also try restarting your database server itself to fix any issues with instability.

Thus:

1. Stop Confluence
2. Restart the database server
3. Start Confluence.

Let us know if you have any trouble!

Kind Regards,
Shannon