Confluence Service crashing frequently

[Sabareesh Gunasekaran]

Hi Team,

We are running confluence server in the Ubuntu 22.04. Until now all perfect. But past few days regularly the confluence not working . once we restart after 20 - 30 minutes again the problem occurs. 

Then we followed the new security advisory email from atlassian and upgraded our version to 17.18.1

https://confluence.atlassian.com/doc/confluence-security-advisory-2022-06-02-1130377146.html

Same server we are running the Bitbucket and Jira. Those are working fine without any issues.

The CPU usage of the service is 85% to 95%. some times uses all cpu. If we stop the confluence the cpu usage is drops to 5 % to 10% . even running the other services Bitbucket and Jira.

The CPU usage is very abnormal for past few days and keep the confluence down after 15 minutes we start. 

Please let us know, what are steps we need to follow to fix the issue. Its production server and affecting our development activities. We are waiting for your valuable reply. 

Confluence Service crashing frequently

[Nic Brough -Adaptavist- · Accepted solution]

Start by monitoring and reading logs.  What are the patterns of usage, and when within them is it failing (note that "it looks random" is a perfectly good answer to that, what we're looking for is patterns.  If there isn't one that correlates with user usage patterns, that's fine, you've probably ruled out a load of stuff as causing the crash)

Your logs are probably the first place to look.  The most interesting ones are going to be the handful of lines before the crash, but the whole thing is worth a look if you've got high CPU usage all the time.  The logs are trying to tell you what the service is actually doing, as well as catching data about failures.

The ones to read first are the application log (<confluence home>/log/atlassian-confluence.log and all the other logs in that directory), then the application server log (<confluence install>/logs/catalina.out )

[Sabareesh Gunasekaran]

Hi Nic,

thanks for your guidance , I checked the confluence logs under

<confluence install>/logs/catalina.out

---

found its a java heap size issue

After I followed the below steps and now the confluence running without any issues.

#Stop Confluence service

cd <confluence install>/bin

./stop-confluence.sh

cd <confluence install>/bin

nano setenv.sh

CATALINA_OPTS="-Xms1024m -Xmx1024m -XX:+UseG1GC ${CATALINA_OPTS}"

to 

CATALINA_OPTS="-Xms2048m -Xmx2048m -XX:+UseG1GC ${CATALINA_OPTS}"

took reference from the below discussion

https://community.atlassian.com/t5/Confluence-questions/How-do-I-fix-OutOfMemoryError-Java-heap-space/qaq-p/1315030  

[Nic Brough -Adaptavist-]

Excellent!  I'm glad it was an easy one to find and fix!

One thing that I do find lacking in Atlassian's docs about installing server is that they don't say (loudly enough) "the default setup is only for a trial, demo or quick test system.  You should review them with a view to making them suitable for your intended use of the system"

I lost count of the installs I've done years ago, but I know I've changed the memory settings on every one I've done since <mumble>  before even thinking of starting the new install up!