Confluence on-prem server shutting down issue

[Kai Gao]

Configured the confluence on-prem server and confluence db on EC2 instance using docker. Please see the below docker-compose.yml

version: '3'

services:
confluence:
image: atlassian/confluence:7.19.0
environment:
- DISABLE_NOTIFICATIONS=TRUE
- CONF_ARGS=-Datlassian.plugins.enable.wait=300
volumes:
- ./data:/opt/atlassian/confluence/data
ports:
- 8090:8090
environment:
- JVM_MINIMUM_MEMORY=1024m
- JVM_MAXIMUM_MEMORY=4096m
- JVM_RESERVED_CODE_CACHE_SIZE=512m
restart: always

confluencedb:
image: postgres:14
environment:
- POSTGRES_PASSWORD=secret
- POSTGRES_USER=confluence
- POSTGRES_DB=confluence
volumes:
- ./db:/var/lib/postgresql/data
ports:
- 5432:5432
restart: always

While running performance testing to insert 500K records(heavy operation) in confluence on-prem server using some script or when the confluence on-prem server is idle, I have observed the confluence on-prem server shutting down daily means when performing heavy operations on confluence on-prem server, the server always shuts down and gets the below error.

Out of memory issue

Exception in thread "I/O dispatcher 11" Exception in thread "I/O dispatcher 17" Exception in thread "I/O dispatcher 7" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
29-Oct-2022 00:08:58.709 SEVERE [Catalina-utility-4] org.apache.catalina.core.StandardServer.startPeriodicLifecycleEvent Error sending periodic event
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)

Despite increasing the heap memory of confluence on-prem server in docker-compose.yml file with - JVM_RESERVED_CODE_CACHE_SIZE=512m, the server is still shutting down.

DB connection pool issue

2-Nov-2022 04:48:43.442 SEVERE [http-nio-8090-exec-48] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=500, location=/500page.jsp]
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is java.lang.IllegalStateException: DB connection pool is exhausted; obtaining a new DB connection for this request is disallowed.
at org.springframework.orm.hibernate.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:476)
at com.atlassian.confluence.impl.hibernate.ConfluenceHibernateTransactionManager.doBegin(ConfluenceHibernateTransactionManager.java:41)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)

After booting up the containers everything works well but again When performing heavy operations on confluence on-prem server, the server shuts down.

It would be great if you could assist me with the following question to resolve the issue with Confluence on-prem server shutting down.

1. When performing heavy operations on confluence on-prem server and the server shuts down then how can we debug/check it?
2. What should be the system requirement to setup the confluence on-prem server and confluence db for running the heavy load?

[Kai Gao]

@Atlassian Support  We are waiting for your response. Could you please look into it and guide me how can we resolve this issue.

[Tomasz Serafinski]

@Kai Gao - were You able to figure out what was the problem?