Ubuntu 18.04 Confluence 6.15.3 setup db problem

[EdisonCh]

Dear All,

I have tried more than 6 times to install confluence 6.15.3 on Ubuntu 18.04 using Postgresql version 10.7 and it always crashed on same place with errors below.

I have increase memory of catalina of confluence to 4096m for Xmx and Xms. I have repeatly drop database in postgresql and recreate them using encoding UTF8; however, all my efforts failed. I am running out of idea of how to make confluence runs in my server. It is physical server with more than enough RAM (32 GB ECC) and 2 CPU of E5-2609 v2 @ 2.50GHz. I have more than enough hardware specs to run confluence for first time installation. Why is it always failing with the same error below.

I used this command in psql to create database in postgresql:

CREATE DATABASE "confluence" WITH OWNER "confluenceuser" ENCODING 'UTF8' LC_COLLATE = 'en_US.utf8' LC_CTYPE = 'en_US.utf8' TEMPLATE template0;

Please help.

------

WARNING [Catalina-utility-1] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread [http-nio-9777-exec-7] (id=[68]) has been active for [69,707] milliseconds (since [5/1/19 12:48 AM]) to serve the same request for [http://geminiwest:9777/confluence/setup/setupdbtype.action] and may be stuck (configured threshold for this StuckThreadDetectionValve is [60] seconds). There is/are [1] thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1354)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:292)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
at org.postgresql.jdbc.TypeInfoCache.getSQLType(TypeInfoCache.java:225)
at org.postgresql.jdbc.PgDatabaseMetaData.getTypeInfo(PgDatabaseMetaData.java:2173)
at com.mchange.v2.c3p0.impl.NewProxyDatabaseMetaData.getTypeInfo(NewProxyDatabaseMetaData.java:3326)
at org.hibernate.engine.jdbc.spi.TypeInfo.extractTypeInfo(TypeInfo.java:84)
at org.hibernate.engine.jdbc.env.internal.ExtractedDatabaseMetaDataImpl$Builder.apply(ExtractedDatabaseMetaDataImpl.java:194)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentImpl.<init>(JdbcEnvironmentImpl.java:231)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:114)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)

....

01-May-2019 00:49:58.892 WARNING [Catalina-utility-3] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadCompleted Thread [http-nio-9777-exec-7] (id=[68]) was previously reported to be stuck but has completed. It was active for approximately [81,124] milliseconds.

[EdisonCh · Accepted solution]

After reading more questions in atlassian community using another keyword 504 Gateway Timeout, I applied their answers to increase timeout for NGINX and database query from confluence to database - my main problem is solved. 

It happens confluence despite monopolize 2 Xeon CPU and has 4 GB memory at its disposal would need more than 2 minutes to finish setup the database at confluence setting.

The fixes would be in two files (assuming you are using nginx as front end as https and then proxy to confluence at back end as http).

This would assume you would have nginx server block https://confluence.yourdomain.com.

Here is your confluence.conf need to be put into <nginx-configuration-location>/conf.d/confluence.conf

server {
listen 443;
server_name confluence.yourdomain.com;
client_max_body_size 200M;

ssl on;
ssl_certificate /etc/ssl/certificates/your-confluence-pem-file
ssl_certificate_key /etc/ssl/certificates/your-confluence-key-file
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

# Fix 'The Logjam Attack'.
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-EC
DSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:
ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES2
56-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-
SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3
-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CB
C3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-
CBC3-SHA;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/ssl/dhparams.pem;

location / {
add_header Front-End-Https on;
add_header Cache-Control "public, must-revalidate";
add_header Strict-Transport-Security "max-age=2592000; includeSubdomains";

# this is important to increase proxy timeout to backend otherwise you would get 504 gateway timeout

proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;

proxy_pass http://confluence.your-backend-server.com:9777;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

}

location /synchrony {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_pass http://confluence.your-backend-server.com:9777/synchrony;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}

include /usr/local/etc/nginx/templates/hsts.tmpl;

}

Second step is to increase database query timeout between confluence with its database.

You would need to extract jar file of confluence-xx-xx-xx.jar (xx-xx-xx is your confluence version) to edit databaseSubsystemContext.xml inside of that jar file.

confluence-xx-xx.jar file is located in /opt/atlassian/confluence/confluence/WEB-INF/lib

You would copy that jar file to another place and execute java -xvf confluence-xx-xx-xx.jar jartemp/ 

Then in jartemp/ you would need to find databaseSubsystemContext.xml and find this part:

<bean id="tenantedTransactionManager"
class="com.atlassian.confluence.impl.hibernate.ConfluenceHibernateTransactionManager">

inside that bean tag add this part before its closing tag </bean>:

<property name="defaultTimeout" value="360"/>

Save the edit xml file then copy back to /opt/atlassian/confluence/confluence/WEB-INF/classes so confluence would be run with this particular xml to override the same file inside of the jar file in lib/ folder.

Then you are done.

If by any chance you still get 504 Gateway timeout, do increase defaultTimeout value first from 360 (seconds) to higher value along with nginx proxy timeout higher than 600 (seconds).

Good luck.