confluence stopped working with a fatal error

[Mana Taghdiri]

we have an on-premise installation of confluence, which has been working perfectly well for the past 2 years. Today, when I tried to access our confluence page, I see the following error message, that complains about unexpectedRollbackException

I tried to restart the confluence server, but still get the same error. atlassian-confluence.log for startup is given below - any help will be appreciated!

2018-06-04 12:56:09,881 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 5.10.7 [build 6441 based on commit hash 08fdc84b52e6e9048619981d39d86d9bfcc50916]
2018-06-04 12:56:13,597 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization started
2018-06-04 12:56:18,912 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] <init> Loading EhCache cache manager
2018-06-04 12:56:21,391 WARN [localhost-startStop-1] [org.springframework.beans.GenericTypeAwarePropertyDescriptor] getWriteMethodForActualAccess Invalid JavaBean property 'cacheFactory' being accessed! Ambiguous write methods found next to actually used [public void com.atlassian.confluence.spaces.persistence.dao.hibernate.HibernateSpaceDao.setCacheFactory(com.atlassian.vcache.VCacheFactory)]: [public void com.atlassian.confluence.core.persistence.hibernate.VersionedHibernateObjectDao.setCacheFactory(com.atlassian.cache.CacheFactory)]
2018-06-04 12:56:39,851 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization completed in 26254 ms
2018-06-04 12:56:48,995 INFO [localhost-startStop-1] [atlassian.plugin.manager.DefaultPluginManager] earlyStartup Plugin system earlyStartup begun
2018-06-04 12:57:15,956 WARN [ThreadPoolAsyncTaskExecutor::Thread 7] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:16,488 WARN [ThreadPoolAsyncTaskExecutor::Thread 23] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:22,007 WARN [ThreadPoolAsyncTaskExecutor::Thread 1] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:22,023 WARN [ThreadPoolAsyncTaskExecutor::Thread 20] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:22,037 WARN [ThreadPoolAsyncTaskExecutor::Thread 3] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:22,319 WARN [ThreadPoolAsyncTaskExecutor::Thread 12] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:22,600 WARN [ThreadPoolAsyncTaskExecutor::Thread 20] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:50,601 INFO [localhost-startStop-1] [atlassian.plugin.manager.DefaultPluginManager] logTime Plugin system earlyStartup ended
2018-06-04 12:57:50,644 INFO [localhost-startStop-1] [atlassian.plugin.manager.DefaultPluginManager] lateStartup Plugin system lateStartup begun
2018-06-04 12:57:53,938 WARN [ThreadPoolAsyncTaskExecutor::Thread 1] [spring.scanner.util.ProductFilterUtil] detectProduct Couldn't detect product, will use ProductFilter.ALL
2018-06-04 12:57:57,611 WARN [AtlassianEvent::CustomizableThreadFactory-1] [core.persistence.schema.StartupSchemaChecker] handleWarnings Database schema is inconsistent with expectations. 3 potential issues found, see below for details.
2018-06-04 12:57:57,612 WARN [AtlassianEvent::CustomizableThreadFactory-1] [core.persistence.schema.StartupSchemaChecker] handleWarnings Column [lowerspacekey] of table [spaces] has mismatched definitions for expected [Column spaces.lowerspacekey nvarchar2(255) nullable] and actual [Column spaces.lowerspacekey nvarchar2(255) non-nullable]
2018-06-04 12:57:57,612 WARN [AtlassianEvent::CustomizableThreadFactory-1] [core.persistence.schema.StartupSchemaChecker] handleWarnings Column [lowerurl] of table [extrnlnks] has mismatched definitions for expected [Column extrnlnks.lowerurl nvarchar2(255) nullable] and actual [Column extrnlnks.lowerurl nvarchar2(255) non-nullable]
2018-06-04 12:57:57,612 WARN [AtlassianEvent::CustomizableThreadFactory-1] [core.persistence.schema.StartupSchemaChecker] handleWarnings Column [lowerurl] of table [trackbacklinks] has mismatched definitions for expected [Column trackbacklinks.lowerurl nvarchar2(255) nullable] and actual [Column trackbacklinks.lowerurl nvarchar2(255) non-nullable]
2018-06-04 12:58:07,424 INFO [localhost-startStop-1] [atlassian.plugin.manager.DefaultPluginManager] logTime Plugin system lateStartup ended
2018-06-04 12:58:13,742 INFO [localhost-startStop-1] [confluence.upgrade.impl.DefaultUpgradeGate] waitForBooleanValue Waiting to find if plugin dependent upgrades are required. Maximum wait time will be 90 seconds.
2018-06-04 12:58:13,743 INFO [localhost-startStop-1] [confluence.upgrade.impl.DefaultUpgradeGate] waitForBooleanValue plugin dependent upgrades are required : false.
2018-06-04 12:58:16,234 ERROR [localhost-startStop-1] [sf.hibernate.util.JDBCExceptionReporter] logExceptions ORA-01555: snapshot too old: rollback segment number with name "" too small
ORA-22924: snapshot too old

2018-06-04 12:58:16,235 ERROR [localhost-startStop-1] [sf.hibernate.util.JDBCExceptionReporter] logExceptions ORA-01555: snapshot too old: rollback segment number with name "" too small
ORA-22924: snapshot too old

2018-06-04 12:58:16,237 WARN [localhost-startStop-1] [confluence.impl.vcache.SynchronousExternalCache] lambda$get$11 Failed to read entry from cache 'com.atlassian.bandana.BandanaPersister': Failed due to UNCLASSIFIED_FAILURE
2018-06-04 12:58:16,273 WARN [localhost-startStop-1] [confluence.setup.bandana.ConfluenceDaoBandanaPersister] getObjectFromValue Configuration could not be loaded because class could not be found (context: null, key: null).
java.lang.NullPointerException: null
java.lang.NullPointerException
at java.io.StringReader.<init>(StringReader.java:50)
at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.getObjectFromValue(ConfluenceDaoBandanaPersister.java:100)
at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:30)
at sun.reflect.GeneratedMethodAccessor238.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy34.retrieve(Unknown Source)
at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.lambda$retrieve$0(ConfluenceCachingBandanaPersister.java:80)
at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.lambda$get$11(SynchronousExternalCache.java:239)
at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.lambda$foldResult$0(CompletionStageUtils.java:19)
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
at java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
at java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
at com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
at com.atlassian.confluence.impl.vcache.SynchronousExternalCache.get(SynchronousExternalCache.java:236)
at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:78)
at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:71)
at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:24)
at com.atlassian.confluence.schedule.persistence.dao.CachedScheduledJobDao.getScheduledJobHistory(CachedScheduledJobDao.java:124)
at com.atlassian.confluence.schedule.quartz.ManagedScheduledJobTriggerFactory.createAndRegisterTrigger(ManagedScheduledJobTriggerFactory.java:168)
at com.atlassian.confluence.schedule.quartz.ManagedScheduledJobTriggerFactory.initialiseTriggers(ManagedScheduledJobTriggerFactory.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy132.initialiseTriggers(Unknown Source)
at com.atlassian.confluence.schedule.listeners.ManagedScheduledJobsLifecycle.startup(ManagedScheduledJobsLifecycle.java:24)
at com.atlassian.config.lifecycle.DefaultLifecycleManager.startUp(DefaultLifecycleManager.java:40)
at com.atlassian.config.lifecycle.LifecycleServletContextListener.contextInitialized(LifecycleServletContextListener.java:17)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4842)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5303)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1407)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1397)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-06-04 12:58:16,331 ERROR [localhost-startStop-1] [atlassian.config.lifecycle.LifecycleManager] panicAndShutdown Unable to start up Confluence. Fatal error during startup sequence: confluence.lifecycle.core:mananagedjobs (Start and stop the Managed Scheduled Jobs) - org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only
org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy32.commit(Unknown Source)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy132.initialiseTriggers(Unknown Source)
at com.atlassian.confluence.schedule.listeners.ManagedScheduledJobsLifecycle.startup(ManagedScheduledJobsLifecycle.java:24)
at com.atlassian.config.lifecycle.DefaultLifecycleManager.startUp(DefaultLifecycleManager.java:40)
at com.atlassian.config.lifecycle.LifecycleServletContextListener.contextInitialized(LifecycleServletContextListener.java:17)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4842)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5303)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1407)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1397)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

[Shannon S · Accepted solution]

Hi there,

It seems like you may be running into this issue:

• Confluence Fails with ORA-01555 Error

Can you review that article with possible resolutions and let us know if you have any trouble? It will also help if you let us know your exactly Confluence and Oracle versions.

Regards,

Shannon

[Mana Taghdiri]

Hi Shannon,

Thanks for your quick reply, and for sending me the link. The steps worked perfectly well, and our confluence is up and running again.

Regards,

Mana

[Shannon S]

Great news, Mana! Glad you got Confluence up and running again. :) 

Take care,

Shannon