nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Confluence web site does not load after restart

Christian Schilling_qig4b4

Hello ladies or gents,

after a recent migration to a VM I was happy that confluence 6.5.1 started up nicely with the previous data imported.

Unfortunately confluence web pages do not load anymore after I restarted confluence. the last changes I made are:

Installed PlaySQL plugin (incl. change in server.xml, which i commented out again)
fixed hostname DNS lookup issue by adding static information to /etc/hosts (like "x.x.x.x SERVER.LAN SERVER")

I cannot find the reason in catalina.out or atlassian-confluence.log for not serving web pages anymore.

I am also running JIRA on the same machine and that runs nicely as before.

Here some data on the machine:

Hyper-V VM running Ubuntu
8GB RAM
Confluence 6.5.1
PostgreSQL

Some excerpts from the atlassian-confluence.log:

2017-12-07 15:40:25,184 WARN [synchrony-interop-executor:thread-2] [plugins.synchrony.bootstrap.DefaultSynchronyProxyMonitor] pollHealthcheck Could not ping the synchrony-proxy [http://127.0.0.1:8090/synchrony-proxy/healthcheck]: {}
java.net.SocketTimeoutException: Read timed out
   at java.net.SocketInputStream.socketRead0(Native Method)
   at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   at java.net.SocketInputStream.read(SocketInputStream.java:171)
   at java.net.SocketInputStream.read(SocketInputStream.java:141)
   at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
   at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
   at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
   at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyProxyMonitor.pollHealthcheck(DefaultSynchronyProxyMonitor.java:76)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)

...

2017-12-07 15:40:33,378 WARN [ListenableFutureAdapter-thread-2] [plugins.synchrony.bootstrap.DefaultSynchronyMonitor] lambda$null$3 Rescheduled heartbeat check failed.
com.atlassian.util.concurrent.TimedOutException: Timed out after: 33 SECONDS
   at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyMonitor.lambda$pollHeartbeat$4(DefaultSynchronyMonitor.java:96)
   at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyMonitor.pollHeartbeat(DefaultSynchronyMonitor.java:119)
   at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyMonitor.lambda$null$1(DefaultSynchronyMonitor.java:102)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
2017-12-07 15:40:33,382 WARN [ListenableFutureAdapter-thread-2] [plugins.synchrony.config.DefaultSynchronyConfigurationManager] enableSharedDrafts [Collab editing plugin] Enabling Shared Drafts

...

------

I appreciate ideas of what I can check or fix to repair my confluence.

Thanks in advance!

Christian

Accepted answers

All comments

Shannon S

Hi Christian,

The messages you displayed are healthcheck messages. When you restart Confluence, can you tail the logs and if you see any messages that start with ERROR, can you let us know what those are?

Please also confirm your Postgres version.

Thank you!

Regards,

Shannon

Christian Schilling_qig4b4

Hello Shannon,

thank you for your response.

I tried to follow the logs (atlassian-confluence.log and catalina.out) and see if any SEVERE or ERROR statements come up.

I am running on Postgres 9.5.10.

I recently added PlaySQL but did not finish configuration. I see these types of ERROR messages:

 
[...]
2017-12-07 15:25:16,073 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions Connections could not be acquired from the underlying database!
[...]
2017-12-07 15:25:16,074 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions An SQLException was provoked by the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A ResourcePool cannot acquire a new resource -- the factory or source appears to be down.2017-12-07 15:25:40,045 ERROR [C3P0PooledConnectionPoolManager[identityToken->s0uznlqote5wrssf5b|69a92d62]-HelperThread-#2] [org.postgresql.Driver] connect Connection error:
[...]
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.postgresql.core.PGStream.<init>(PGStream.java:68)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
    ... 13 more
[...]
2017-12-07 15:25:34,701 ERROR [FelixShutdown] [internal.util.concurrent.RunnableTimedExecution] execute Closing runnable for context NonValidatingOsgiBundleXmlApplicationContext(bundle=com.atlassian.upm.atlassian-universal-plugin-manager-plugin, config=osgibundle:/META-INF/spring/*.xml) did not finish in 10000ms; consider taking a snapshot and then shutdown the VM in case the thread still hangs

Christian Schilling_qig4b4

Hello Shannon,

thank you for your response.

I tried to follow the logs (atlassian-confluence.log and catalina.out) and see if any SEVERE or ERROR statements come up.

I am running on Postgres 9.5.10.

I recently added PlaySQL but did not finish configuration. I see these types of ERROR messages:

[...]
2017-12-07 15:25:16,073 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions Connections could not be acquired from the underlying database!
[...]
2017-12-07 15:25:16,074 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions An SQLException was provoked by the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A ResourcePool cannot acquire a new resource -- the factory or source appears to be down.2017-12-07 15:25:40,045 ERROR [C3P0PooledConnectionPoolManager[identityToken->s0uznlqote5wrssf5b|69a92d62]-HelperThread-#2] [org.postgresql.Driver] connect Connection error:
[...]
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.postgresql.core.PGStream.<init>(PGStream.java:68)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
    ... 13 more
[...]
2017-12-07 15:25:34,701 ERROR [FelixShutdown] [internal.util.concurrent.RunnableTimedExecution] execute Closing runnable for context NonValidatingOsgiBundleXmlApplicationContext(bundle=com.atlassian.upm.atlassian-universal-plugin-manager-plugin, config=osgibundle:/META-INF/spring/*.xml) did not finish in 10000ms; consider taking a snapshot and then shutdown the VM in case the thread still hangs

Christian Schilling_qig4b4

Hello Shannon,

thank you for your response.

I tried to follow the logs (atlassian-confluence.log and catalina.out) and see if any SEVERE or ERROR statements come up.

I am running on Postgres 9.5.10.

I recently added PlaySQL but did not finish configuration. I see these types of ERROR messages:

[...]

2017-12-07 15:25:16,073 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions Connections could not be acquired from the underlying database!
[...]
2017-12-07 15:25:16,074 ERROR [Gemini Blueprint context shutdown thread2] [engine.jdbc.spi.SqlExceptionHelper] logExceptions An SQLException was provoked by the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A ResourcePool cannot acquire a new resource -- the factory or source appears to be down.2017-12-07 15:25:40,045 ERROR [C3P0PooledConnectionPoolManager[identityToken->s0uznlqote5wrssf5b|69a92d62]-HelperThread-#2] [org.postgresql.Driver] connect Connection error:
[...]
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.postgresql.core.PGStream.<init>(PGStream.java:68)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
    ... 13 more
[...]
2017-12-07 15:25:34,701 ERROR [FelixShutdown] [internal.util.concurrent.RunnableTimedExecution] execute Closing runnable for context NonValidatingOsgiBundleXmlApplicationContext(bundle=com.atlassian.upm.atlassian-universal-plugin-manager-plugin, config=osgibundle:/META-INF/spring/*.xml) did not finish in 10000ms; consider taking a snapshot and then shutdown the VM in case the thread still hangs

Nic Brough -Adaptavist-

The first line of your error says it all:

org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

Means simply that Confluence cannot get to the database, or that the database is not allowing the connection.

Check the database is running and that you can connect to it, then restart Confluence.

A common way this happens is when the database and Confluence are on the same machine and it restarts - they both start up ok on boot, but the database is not ready when Confluence needs it, hence it fails, but everything looks ok after the startup finishes everything.

Christian Schilling_qig4b4

Hello Nic,

thank you for the suggestion. I restarted Confluence while JIRA was still running on the same server connected to the same DB (which is running on the same machine).

I entered invalid credentials in the PlaySQL plugin, I believe. That is probably why the plugin reports errors.

Running "ps ax|grep confluence" I can see that confluence itself is running and I see some DB connection processes, just like JIRA. I do not see a response to web requests, not even an error message from the web server. I do not see anything like "ERROR" or "SEVERE" in catalina,out. Should I be looking at something other than that?

Btw: Since JIRA Software and Confluence are linked, JIRA reports "Application Link Failure" ...

Christian Schilling_qig4b4

Here is a temporary link to the files (for 7 days):

Nic Brough -Adaptavist-

Forget the playsql add-on, it's irrelevant to the start up of Confluence, as is the application link. Both of these are things that Confluence looks at later in the start up and it's failing before it gets to them.

You are not getting a response when you browse to Confluence because it's not able to complete the start up. You'll need to read the logs to see what is failing. Is it still the database error, or something else?