Confluence now shows whitepage

[Travis Sheldon]

I've had JIRA and Confluence running for about two weeks. Today Confluence wasn't responding. After a restart of the service (and server) it takes a few minutes for the service to finish booting but it eventually just produces a white screen in the browser.

The context path is set to "" in the server.xml

Remote Address:
10.1.10.16:8090

Request URL:
http://confluence1.internal.boomcarding.com:8090/

Request Method:
GET

Status Code:
404 Not Found


Response Headersview source
Content-Length:
0

Date:
Mon, 06 Jul 2015 14:56:40 GMT

Server:
Apache-Coyote/1.1

catalina.out

06-Jul-2015 10:53:29.026 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal Error filterStart
06-Jul-2015 10:53:29.026 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal Context [] startup failed due to previous errors
06-Jul-2015 10:53:29.258 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [org.h2.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
06-Jul-2015 10:53:29.258 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [org.hsqldb.jdbc.JDBCDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
06-Jul-2015 10:53:29.258 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [org.postgresql.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
06-Jul-2015 10:53:29.259 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:41)
06-Jul-2015 10:53:29.259 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.260 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.260 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.261 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-4] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.261 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-5] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.262 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-6] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.262 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-7] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.263 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-8] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.263 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-9] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.264 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [scheduler_Worker-10] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:543)
06-Jul-2015 10:53:29.264 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.util.TimerThread.mainLoop(Timer.java:552)
 java.util.TimerThread.run(Timer.java:505)
06-Jul-2015 10:53:29.265 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534)
06-Jul-2015 10:53:29.270 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534)
06-Jul-2015 10:53:29.271 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534)
06-Jul-2015 10:53:29.273 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Timer-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.util.TimerThread.mainLoop(Timer.java:552)
 java.util.TimerThread.run(Timer.java:505)
06-Jul-2015 10:53:29.274 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534)
06-Jul-2015 10:53:29.274 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Thread.sleep(Native Method)
 com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1805)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
06-Jul-2015 10:53:29.275 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:534)
06-Jul-2015 10:53:29.282 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8090"]
06-Jul-2015 10:53:29.288 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 133825 ms

[Travis Sheldon]

I found the issue.. the remote database server was locked up and the MSQL connector was failing. 

[mdejonge]

I have the same problem but I'm using the build-in database. Can anyone help?

[Travis Sheldon]

Can you attach the log files? catalina.out doesn't show much so you can skip that. What happens when you run confluence in the foreground and not as a service?

[mdejonge]

After starting confluence I get this message: 25-Aug-2015 17:17:44.320 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file In /var/atlassian/application-data/confluence/logs/atlassian-confluence.log I find this: 2015-08-25 17:21:49,494 FATAL [HSQLDB Connection @1aac6734] [hsqldb.db.HSQLDB4B12203FD3.ENGINE] logSevereEvent readExistingData failed org.hsqldb.HsqlException: integrity constraint violation: unique constraint or index violation; BLOCKS_PK table: BLOCKS Can this perhaps be the cause of the problem?

[Travis Sheldon]

That would cause the white screen because your db adapter isn't initializing without error. What recently changed? Did you import data? It's complaining about the blocks schema having an invalid foreign constraint

[mdejonge]

I didn't do anything special that I now of. Suddenly the system stopped working. I think that was due to an out-of-memory issue. Since then the system doesn't start anymore. I then upgraded from 5.8.5 to 5.8.9 and that's about it. I certainly did not do any import!

[Travis Sheldon]

The version upgrade could have changed the schema. If you don't have a backup that you can try to restore from then you may have to contact technical support as you are in a very broken state. When I run into issues like this I learned its best to solve the problem without doing updates if possible. If you did an update on a broken state then it could have messed up your key constraints

[mdejonge]

That doesn't look good! It seems like the daily backups made by confluence do not include a backup of the internal database. Is this correct? The database file it self (confluencedb.lobs) does not seem to be altered by any update. It has a modification date of august 18th, which was before the update. The other database files (such as the log file confluencedb.log) have a later modification date. Could you be so kind to bring me into contact with technical support?

[mdejonge]

Hi Travis, see message above. My site is completely broken now. can you direct me to technical support?

[Travis Sheldon]

As far as I know you need to have an active support contract with Atlassian which you can then open a support ticket @ https://www.atlassian.com/resources/support Im not sure where the BLOCKS_PK lives (in terms of database and table) so I wouldn't feel comfortable trying to diagnose it with you. Best of luck with the support ticket and if you get an answer Im sure anyone who comes across this thread would love to know a resolution.

[Merijn de Jonge]

Thanks. For reference: https://support.atlassian.com/servicedesk/customer/portal/14/CSP-155748

[Merijn de Jonge]

I did a fresh install of confluence and during the installation I used a backup from application-data/confluence/backups from the corrupted installation to import my old data. Everything seems to be working correctly again!