Confluence Server doesn't start

[Дмитрий_Заец]

Hello!

My own confluence server doesn't start!

2018-07-12 01:34:57,164 WARN [Gemini Blueprint context shutdown thread1] [beans.factory.support.DisposableBeanAdapter] destroy Invocation of destroy method failed on bean with name 'defaultLogScanService': java.lang.NullPointerException
2018-07-12 01:35:05,248 WARN [C3P0PooledConnectionPoolManager[identityToken->1hgekg19w12oihjf1v2gkqq|2a2d9f80]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5fb0071f -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
org.h2.jdbc.JdbcSQLException: í󲰥��颪ຠ"java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341"
General error: "java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341" [50000-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.convert(DbException.java:294)
at org.h2.table.RegularTable.addRow(RegularTable.java:138)
at org.h2.store.PageStore.redo(PageStore.java:1575)
at org.h2.store.PageLog.recover(PageLog.java:329)
at org.h2.store.PageStore.recover(PageStore.java:1407)
at org.h2.store.PageStore.openExisting(PageStore.java:368)
at org.h2.store.PageStore.open(PageStore.java:289)
at org.h2.engine.Database.getPageStore(Database.java:2366)
at org.h2.engine.Database.open(Database.java:657)
at org.h2.engine.Database.openDatabase(Database.java:260)
at org.h2.engine.Database.<init>(Database.java:254)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:150)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341
at org.h2.message.DbException.throwInternalError(DbException.java:241)
at org.h2.table.RegularTable.checkRowCount(RegularTable.java:170)
at org.h2.table.RegularTable.addRow(RegularTable.java:121)
... 15 more

at org.h2.engine.SessionRemote.done(SessionRemote.java:606)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:129)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:430)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:311)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:74)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-07-12 01:35:05,248 WARN [C3P0PooledConnectionPoolManager[identityToken->1hgekg19w12oihjf1v2gkqq|2a2d9f80]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log Having failed to acquire a resource, com.mchange.v2.resourcepool.BasicResourcePool@441acd25 is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2018-07-12 01:35:09,189 WARN [C3P0PooledConnectionPoolManager[identityToken->1hgekg19w12oihjf1v2gkqq|550039d7]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@693ff323 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
org.h2.jdbc.JdbcSQLException: í󲰥��颪ຠ"java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341"
General error: "java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341" [50000-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.convert(DbException.java:294)
at org.h2.table.RegularTable.addRow(RegularTable.java:138)
at org.h2.store.PageStore.redo(PageStore.java:1575)
at org.h2.store.PageLog.recover(PageLog.java:329)
at org.h2.store.PageStore.recover(PageStore.java:1407)
at org.h2.store.PageStore.openExisting(PageStore.java:368)
at org.h2.store.PageStore.open(PageStore.java:289)
at org.h2.engine.Database.getPageStore(Database.java:2366)
at org.h2.engine.Database.open(Database.java:657)
at org.h2.engine.Database.openDatabase(Database.java:260)
at org.h2.engine.Database.<init>(Database.java:254)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:150)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341
at org.h2.message.DbException.throwInternalError(DbException.java:241)
at org.h2.table.RegularTable.checkRowCount(RegularTable.java:170)
at org.h2.table.RegularTable.addRow(RegularTable.java:121)
... 15 more

at org.h2.engine.SessionRemote.done(SessionRemote.java:606)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:129)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:430)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:311)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:74)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-07-12 01:35:09,190 WARN [C3P0PooledConnectionPoolManager[identityToken->1hgekg19w12oihjf1v2gkqq|550039d7]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log Having failed to acquire a resource, com.mchange.v2.resourcepool.BasicResourcePool@602fa8ab is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2018-07-12 01:35:09,190 ERROR [localhost-startStop-1] [engine.jdbc.spi.SqlExceptionHelper] logExceptions Connections could not be acquired from the underlying database!

[AnnWorley]

Hi - It looks like Confluence cannot acquire enough database connections:

Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30)

Connections could not be acquired from the underlying database!

Please try increasing the database connection pool in <confluence_home>confluence.cfg.xml file. Change 30 to 60 in the following line:

 <property name="hibernate.c3p0.max_size">30</property>

Afterward, Confluence will need to be restarted to pick up the change.

If that doesn't help, please try the checklist on Connections could not be acquired from the underlying database

The log refers to the evaluation H2 database. Please migrate to an external database before putting the instance into Production: Embedded H2 Database

Thanks!

[Дмитрий_Заец]

No, problem in H2 database.

What you wrote about is not the cause of the problem - this is a consequence. I mean this:

Connections could not be acquired from the underlying database!

Problem (why connections are not enough) Is here: 

"java.lang.RuntimeException: rowCount expected 829892 got 27 T95.I341" - database corrupted.

I get same error when I try to connect directly to the database through my SQL client. I create only one connection and confluence was stopped.

[AnnWorley]

You are right, I should have scrolled further down in the error. Thanks for testing with the SQL client to narrow down the issue.

The only way I know to recover a corrupt H2 database is to restore from backup. Your file system backup of the Confluence Home directory should contain the database. Another option, if the default XML backups are being made nightly, you could install a new instance of the same version and import the XML backup. This article has the location of the XML backups and how to restore: Site Backup and Restore

[Дмитрий_Заец]

Thank you! 

I will try restore from backup.