[Crowd] Does not start-up anymore (Initializing c3p0 pool)

[Lucas Molenaar]

Our staging Crowd instance is no longer starting up, not sure what happend. In the log I can see it is waiting on step "Initializing c3p0 pool", see the log below. When I remove the crowd.cfg.xml file I can reinstall crowd again but ofcourse this is not something I like to do.

Could someone advise how to debug my problem?

Mar 11, 2014 3:33:09 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 471 ms
Mar 11, 2014 3:33:09 PM org.apache.catalina.realm.JAASRealm setContainer
INFO: Set JAAS app name Catalina
Mar 11, 2014 3:33:09 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Mar 11, 2014 3:33:09 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32
Mar 11, 2014 3:33:09 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor openidserver.xml
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/crowd_app/atlassian-crowd-2.5.0/crowd-openidserver-webapp/WEB-INF/lib/slf4j-jcl-1.0.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/crowd_app/atlassian-crowd-2.5.0/crowd-openidserver-webapp/WEB-INF/lib/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2014-03-11 15:33:12,779 main INFO [com.atlassian.crowd.startup] Starting Crowd OpenID Server, Version: 2.5.0 (Build:#582 - 02-08-2012)

*********************************************************************************************
*
*  You can now use the Crowd OpenID server by visiting http://localhost:8095/openidserver
*
*********************************************************************************************

2014-03-11 15:33:13,467 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'application.name' : 'crowd-openid-server'
2014-03-11 15:33:13,467 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'application.login.url' : 'http://localhost:8095/openidserver'
2014-03-11 15:33:13,467 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'session.tokenkey' : 'session.tokenkey'
2014-03-11 15:33:13,467 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'session.lastvalidation' : 'session.lastvalidation'
2014-03-11 15:33:13,467 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'session.validationinterval' : '0'
2014-03-11 15:33:13,469 main INFO [service.soap.client.SoapClientPropertiesImpl] Loading property: 'crowd.server.url' : 'http://localhost:8095/crowd/services/'
2014-03-11 15:33:14,086 main WARN [org.openid4java.server.RealmVerifier] RP discovery / realm validation disabled;
Mar 11, 2014 3:33:14 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor openidclient.xml
2014-03-11 15:33:15,408 main WARN [org.openid4java.server.RealmVerifier] RP discovery / realm validation disabled;
Mar 11, 2014 3:33:15 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor demo.xml
2014-03-11 15:33:16,062 main INFO [crowd.service.client.ClientPropertiesImpl] Loading property: 'application.name' : 'demo'
2014-03-11 15:33:16,063 main INFO [crowd.service.client.ClientPropertiesImpl] Loading property: 'application.login.url' : 'http://localhost:8095/demo'
2014-03-11 15:33:16,063 main INFO [crowd.service.client.ClientPropertiesImpl] Loading property: 'session.tokenkey' : 'session.tokenkey'
2014-03-11 15:33:16,063 main INFO [crowd.service.client.ClientPropertiesImpl] Loading property: 'session.lastvalidation' : 'session.lastvalidation'
2014-03-11 15:33:16,063 main INFO [crowd.service.client.ClientPropertiesImpl] Loading property: 'session.validationinterval' : '0'
Mar 11, 2014 3:33:16 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor crowd.xml
2014-03-11 15:33:23,932 main INFO [ContainerBase.[Catalina].[localhost].[/crowd]] Initializing Spring root WebApplicationContext
2014-03-11 15:33:25,045 main INFO [util.persistence.hibernate.ConfigurableLocalSessionFactoryBean] Building new Hibernate SessionFactory
2014-03-11 15:33:25,109 main INFO [mchange.v2.log.MLog] MLog clients using log4j logging.
2014-03-11 15:33:25,173 main INFO [mchange.v2.c3p0.C3P0Registry] Initializing c3p0-0.9.1.2 [built 21-May-2007 15:04:56; debug? true; trace: 10]
2014-03-11 15:33:25,240 main INFO [v2.c3p0.impl.AbstractPoolBackedDataSource] Initializing c3p0 pool... com.mchange.v2.c3p0.PoolBackedDataSource@22135d53 [ connectionPoolDataSource -> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@e9bdebae [ acquireIncrement -> 1, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 1br6lfc901d9mc2cltasxq|28671c2c, idleConnectionTestPeriod -> 100, initialPoolSize -> 0, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 30, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 30, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 0, nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@5a6ec990 [ description -> null, driverClass -> null, factoryClassLocation -> null, identityToken -> 1br6lfc901d9mc2cltasxq|217576e1, jdbcUrl -> jdbc:oracle:thin:@123.123.123.123:1234:AAAAAA, properties -> {user=******, password=******, SetBigStringTryClob=true} ], preferredTestQuery -> null, propertyCycle -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> false; userOverrides: {} ], dataSourceName -> null, factoryClassLocation -> null, identityToken -> 1br6lfc901d9mc2cltasxq|516a7fd3, numHelperThreads -> 3 ]

[Lucas Molenaar · Accepted solution]

There was a database problem, resolved by our DBA. Not sure what the exact nature of the issue was

[Daniel Borcherding]

Lucas,

If you do a diff on your existing crowd.cfg.xml and a new install what difference do you see?

Is it possible that some wonky characters found their way into that file?

D