Community
Products
Confluence
Questions
Confluence loses connection to Database throughout Day

Confluence loses connection to Database throughout Day

Randomly, Confluence has been having connection drops to the PostgreSQL database.

I receive this error:

Whats interesting is that my JIRA set up is running on the same server and same instance of PostgreSQL (9.6) completely fine.

I can provide all the logs you need.

One solution that ive been doing is going to PGadmin(PostgreSQL interface tool) to check on the Databases and seeing if they're connected, which they are. So I leave PGadmin running and i restart Confluence through the service menu in computer management (Windows 10 pro).

Confluence starts to run, however this is a temporary fix, because after a few hours or a day Confluence loses connection to PostgreSQL(Which is still running)

Below is an image of how Confluence DB is configured through PostgreSQL using PGadmin

Again, you can see in the image provided above that JIRAdb is there and running perfectly fine.

The two logs below are Catalina Log and Application Log

14-Nov-2018 00:12:37.386 SEVERE [http-nio-8090-exec-10] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=404, location=/fourohfour.action]
 java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
    at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
    at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
    at java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
    at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
    at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
    at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
    at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.get(SynchronousExternalCache.java:236)
    at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:78)
    at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:71)
    at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
    at com.atlassian.confluence.themes.DefaultThemeManager.getThemeSettings(DefaultThemeManager.java:73)
    at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalThemeKey(DefaultThemeManager.java:49)
    at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalTheme(DefaultThemeManager.java:42)
    at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:80)
    at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:53)
    at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
    at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getDecorator(PluginDecoratorMapper.java:32)
    at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
    at com.opensymphony.module.sitemesh.mapper.InlineDecoratorMapper.getDecorator(InlineDecoratorMapper.java:41)
    at com.atlassian.confluence.util.profiling.ConfluenceDecoratorSelector.selectDecorator(ConfluenceDecoratorSelector.java:64)
    at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:83)
    at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.confluence.util.UserThreadLocalFilter.doFilter(UserThreadLocalFilter.java:39)
    at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
    at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
    at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)
    at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:175)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
    at com.atlassian.confluence.tenant.TenantGate$1.lambda$create$0(TenantGate.java:43)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
    at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
    at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:344)
    at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:313)
    at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:42)
    at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
    at com.sun.proxy.$Proxy55.retrieve(Unknown Source)
    at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.lambda$retrieve$0(ConfluenceCachingBandanaPersister.java:80)
    at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.lambda$get$11(SynchronousExternalCache.java:239)
    at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.lambda$foldResult$0(CompletionStageUtils.java:19)
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
    ... 70 more

2018-11-13 23:06:47,676 ERROR [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [org.postgresql.Driver] connect Connection error: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-11-13 23:06:47,754 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@72131583 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

2 answers

0 votes

4-Nov-2018 00:12:37.386 SEVERE [http-nio-8090-exec-10] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=404, location=/fourohfour.action]
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
   at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
   at java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.get(SynchronousExternalCache.java:236)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:78)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:71)
   at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
   at com.atlassian.confluence.themes.DefaultThemeManager.getThemeSettings(DefaultThemeManager.java:73)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalThemeKey(DefaultThemeManager.java:49)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalTheme(DefaultThemeManager.java:42)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:80)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:53)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getDecorator(PluginDecoratorMapper.java:32)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.opensymphony.module.sitemesh.mapper.InlineDecoratorMapper.getDecorator(InlineDecoratorMapper.java:41)
   at com.atlassian.confluence.util.profiling.ConfluenceDecoratorSelector.selectDecorator(ConfluenceDecoratorSelector.java:64)
   at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:83)
   at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.UserThreadLocalFilter.doFilter(UserThreadLocalFilter.java:39)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)
   at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
   at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384)
   at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
   at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)
   at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:175)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Thread.java:748)
Caused by: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at com.atlassian.confluence.tenant.TenantGate$1.lambda$create$0(TenantGate.java:43)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:344)
   at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:313)
   at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:42)
   at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:29)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
   at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy55.retrieve(Unknown Source)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.lambda$retrieve$0(ConfluenceCachingBandanaPersister.java:80)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.lambda$get$11(SynchronousExternalCache.java:239)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.lambda$foldResult$0(CompletionStageUtils.java:19)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
   ... 70 more

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

0 votes

Is there a "disconnect" error in the logs, or is it just a sudden "database not reachable" type thing? Are there significant errors in the logs before such a disconnect?

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Under which logs can i find that? I can post the specific logs you're looking for

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

I'd expect the errors to go into the application logs, so <confluence home>/atlassian-confluence.log is the main one. There might be informative errors in <confluence install>/catalina.out too.

Have a look at your system info page to locate those two directories if you don't know them already.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

2018-11-13 23:06:47,676 ERROR [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [org.postgresql.Driver] connect Connection error:
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-11-13 23:06:47,754 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@72131583 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

4-Nov-2018 00:12:37.386 SEVERE [http-nio-8090-exec-10] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=404, location=/fourohfour.action]
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
   at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
   at java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.get(SynchronousExternalCache.java:236)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:78)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:71)
   at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
   at com.atlassian.confluence.themes.DefaultThemeManager.getThemeSettings(DefaultThemeManager.java:73)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalThemeKey(DefaultThemeManager.java:49)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalTheme(DefaultThemeManager.java:42)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:80)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:53)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getDecorator(PluginDecoratorMapper.java:32)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.opensymphony.module.sitemesh.mapper.InlineDecoratorMapper.getDecorator(InlineDecoratorMapper.java:41)
   at com.atlassian.confluence.util.profiling.ConfluenceDecoratorSelector.selectDecorator(ConfluenceDecoratorSelector.java:64)
   at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:83)
   at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.UserThreadLocalFilter.doFilter(UserThreadLocalFilter.java:39)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)
   at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
   at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384)
   at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
   at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)
   at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:175)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Thread.java:748)
Caused by: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at com.atlassian.confluence.tenant.TenantGate$1.lambda$create$0(TenantGate.java:43)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:344)
   at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:313)
   at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:42)
   at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:29)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
   at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy55.retrieve(Unknown Source)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.lambda$retrieve$0(ConfluenceCachingBandanaPersister.java:80)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.lambda$get$11(SynchronousExternalCache.java:239)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.lambda$foldResult$0(CompletionStageUtils.java:19)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
   ... 70 more
018-11-13 23:06:47,676 ERROR [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [org.postgresql.Driver] connect Connection error: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-11-13 23:06:47,754 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@72131583 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

018-11-13 23:06:47,676 ERROR [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [org.postgresql.Driver] connect Connection error:
org.postgresql.util.PSQLException: FATAL: the database system is starting up
   at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
   at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
   at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
   at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
   at org.postgresql.Driver.makeConnection(Driver.java:450)
   at org.postgresql.Driver.connect(Driver.java:252)
   at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
   at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
   at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
   at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
   at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-11-13 23:06:47,754 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z4jp4wl1yl4557|6c2fa18e]-HelperThread-#0] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@72131583 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: FATAL: the database system is starting up
   at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
   at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
   at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
   at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
   at org.postgresql.Driver.makeConnection(Driver.java:450)
   at org.postgresql.Driver.connect(Driver.java:252)
   at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
   at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
   at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
   at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
   at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

14-Nov-2018 00:12:37.386 SEVERE [http-nio-8090-exec-10] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=404, location=/fourohfour.action]
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
   at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
   at java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
   at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.get(SynchronousExternalCache.java:236)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:78)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:71)
   at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
   at com.atlassian.confluence.themes.DefaultThemeManager.getThemeSettings(DefaultThemeManager.java:73)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalThemeKey(DefaultThemeManager.java:49)
   at com.atlassian.confluence.themes.DefaultThemeManager.getGlobalTheme(DefaultThemeManager.java:42)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:80)
   at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:53)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getDecorator(PluginDecoratorMapper.java:32)
   at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)
   at com.opensymphony.module.sitemesh.mapper.InlineDecoratorMapper.getDecorator(InlineDecoratorMapper.java:41)
   at com.atlassian.confluence.util.profiling.ConfluenceDecoratorSelector.selectDecorator(ConfluenceDecoratorSelector.java:64)
   at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:83)
   at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.UserThreadLocalFilter.doFilter(UserThreadLocalFilter.java:39)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
   at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
   at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)
   at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
   at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384)
   at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
   at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)
   at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:175)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.lang.Thread.run(Thread.java:748)
Caused by: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
   at com.atlassian.confluence.tenant.TenantGate$1.lambda$create$0(TenantGate.java:43)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at com.sun.proxy.$Proxy54.getCurrentSession(Unknown Source)
   at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:344)
   at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:313)
   at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:42)
   at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:29)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
   at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
   at com.sun.proxy.$Proxy55.retrieve(Unknown Source)
   at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.lambda$retrieve$0(ConfluenceCachingBandanaPersister.java:80)
   at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.lambda$get$11(SynchronousExternalCache.java:239)
   at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.lambda$foldResult$0(CompletionStageUtils.java:19)
   at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
   ... 70 more

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

I edited the ticket to include the catalina.2018-11-14.log (1st block code)

and the atlassian-confluence.log (2nd block code)

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

Alright, so I see two errors in your logs:

java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.

We see this one fairly regularly, usually on the first time someone tries to start Confluence after a fresh installation. The fix is usually fixing the DNS resolution of postgres on the host where Confluence is running. I generally wouldn't expect this to be what's happening to you over time, unless for some reason the host Confluence is running on intermittently can't get responses back from your DNS server and just dumps its records after their TTL expires. So it is worth testing the diagnosis in this article if you can catch it happening. Or maybe even run a bash script that checks DNS resolution on the Confluence server every minute or two, so you can correlate any Confluence breakages with DNS resolution failure.

The second error:

Connection error:
org.postgresql.util.PSQLException: FATAL: the database system is starting up

Looks like this can happen in Postgres when trying to connect to a slave database when replication is running. Information I was able to find on this suggested making sure the following line was enabled in postgresql.conf (for the replication copy):

hot_standby = on

I'm not sure from your screenshot if you've got a hot backup running, so that also seems worth investigating.

Also take a look to make sure validation queries are enabled (these are only enabled by default on fresh installations after Confluence 6.5 - if you've upgraded from an older version, you have to enable it manually). This will get you past tiny disconnects from the database connector.

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

On postgresql.conf this is what i see

#hot_standby = off

should i turn it on? What does that resolve

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

With the disclaimer that I'm not a Postgresql expert and have not tested this, what I was able to find indicated that this was about still being able to run queries when Postgres is in the process of backing up or making a replication copy.

You can read about this in the actual Postgresql documentation.

Enabling this flag should allow read-only queries to happen. Again, I don't know enough about your database environment to know if you've got Confluence trying to connect to a postgresql slave database. That definitely could be a problem, and if you've got replication enabled (or aren't sure if you don't have it enabled) this would be something you want to track down. If you've got an in-house Postgresql administrator, this would be a good opportunity to buy them lunch.

Cheers,
Daniel

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Comment

I am novice to this so please forgive me if my questions seem very obvious to answer by admin standards. I am frustrated because this has been running along side JIRA for 13 months and im just now receiving these errors.

This is the logs by Postgresql

2018-11-16 08:49:21 EST LOG:  could not receive data from client: An existing connection was forcibly closed by the remote host.

    
2018-11-16 08:49:23 EST LOG:  received fast shutdown request
2018-11-16 08:49:23 EST LOG:  aborting any active transactions
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST FATAL:  terminating connection due to administrator command
2018-11-16 08:49:23 EST LOG:  autovacuum launcher shutting down
2018-11-16 08:49:24 EST ERROR:  canceling statement due to user request
2018-11-16 08:49:24 EST LOG:  autovacuum launcher shutting down
2018-11-16 08:49:38 EST LOG:  shutting down
2018-11-16 08:49:39 EST LOG:  database system is shut down

Here are the latest errors from Confluence

2018-11-16 08:32:19,137 ERROR [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z7yvj7j1c59slr|647f8327]-HelperThread-#1] [org.postgresql.Driver] connect Connection error: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
2018-11-16 08:32:19,184 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvxtv9z7yvj7j1c59slr|647f8327]-HelperThread-#1] [mchange.v2.resourcepool.BasicResourcePool] log com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@58e66429 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: 
org.postgresql.util.PSQLException: FATAL: the database system is starting up
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:450)
    at org.postgresql.Driver.connect(Driver.java:252)
    at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
    at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
    at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
    at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
    at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
    at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Missed Team ’24? Catch up on announcements here.

Products

Community resources

Support

Top groups

Community resources

Support

Learn

Community resources

Support

Events

Community resources

Support

Get product advice from experts

Join a community group

Advance your career with learning paths

Earn badges and rewards

Connect and share ideas at events

Confluence loses connection to Database throughout Day

2 answers

Suggest an answer

Was this helpful?

Thanks!

TAGS

Atlassian Community Events

Missed Team ’24? Catch up on announcements here.

Ask a question

Products

Community resources

Support

Top groups

Community resources

Support

Learn

Community resources

Support

Events

Community resources

Support

Get product advice from experts

Join a community group

Advance your career with learning paths

Earn badges and rewards

Connect and share ideas at events

Confluence loses connection to Database throughout Day

2 answers

Suggest an answer

Was this helpful?

Thanks!

TAGS

Atlassian Community Events