Hazelcast instance is not active!

[Oday Rafeh]

Hi There, 

We just run into an issue with our software and this error coming directly to all users its very important to solve this as soon as possible because no one can use the software right now. 

HTTP Status 500 – Internal Server Error

---

Type Exception Report

Message Hazelcast instance is not active!

Description The server encountered an unexpected condition that prevented it from fulfilling the request.

Exception

com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
	com.hazelcast.instance.DefaultNodeExtension$1.get(DefaultNodeExtension.java:205)
	com.hazelcast.instance.DefaultNodeExtension$1.get(DefaultNodeExtension.java:202)
	com.hazelcast.internal.serialization.impl.bufferpool.BufferPoolThreadLocal.get(BufferPoolThreadLocal.java:107)
	com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:141)
	com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:133)
	com.hazelcast.internal.serialization.impl.AbstractSerializationService.toData(AbstractSerializationService.java:118)
	com.hazelcast.map.impl.proxy.MapProxySupport.toDataWithStrategy(MapProxySupport.java:1262)
	com.hazelcast.map.impl.proxy.NearCachedMapProxyImpl.getInternal(NearCachedMapProxyImpl.java:113)
	com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:120)
	com.atlassian.cache.hazelcast.HazelcastCache.getOrLoad(HazelcastCache.java:308)
	com.atlassian.cache.hazelcast.HazelcastCache.get(HazelcastCache.java:125)
	com.atlassian.confluence.cache.DefaultConfluenceCache.get(DefaultConfluenceCache.java:53)
	com.atlassian.confluence.cache.ConfluenceMonitoringCache.get(ConfluenceMonitoringCache.java:57)
	com.atlassian.confluence.impl.cache.ReadThroughAtlassianCache.get(ReadThroughAtlassianCache.java:88)
	com.atlassian.confluence.impl.cache.ReadThroughCache.get(ReadThroughCache.java:28)
	com.atlassian.confluence.impl.bandana.ReadThroughCachingBandanaPersister.retrieve(ReadThroughCachingBandanaPersister.java:82)
	com.atlassian.confluence.impl.bandana.ReadThroughCachingBandanaPersister.retrieve(ReadThroughCachingBandanaPersister.java:75)
	com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
	com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:24)
	com.atlassian.confluence.setup.settings.DefaultSettingsManager.getGlobalSettings(DefaultSettingsManager.java:40)
	com.atlassian.confluence.languages.DefaultLocaleManager.localeValidAndSaveable(DefaultLocaleManager.java:279)
	com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:267)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.lambda$null$2(LanguageExtractionFilter.java:37)
	java.base/java.util.Optional.ifPresent(Unknown Source)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.lambda$doFilter$3(LanguageExtractionFilter.java:37)
	java.base/java.util.Optional.ifPresent(Unknown Source)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:37)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:84)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:68)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:67)
	com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
	com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
	com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
	com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
	com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
	com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:42)
	com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Note The full stack trace of the root cause is available in the server logs.

[G subramanyam · Accepted solution]

Hi @Oday Rafeh welcome to the Atlassian community.

I have found this thread similar to your issue. Beside, I would request you to log a request with Atlassian support team. under technical issues & bugs.

[Oday Rafeh]

Thanks for your Help 

I found that the directory 

/opt/Atlassian/confluence/bin/ 

does not exist and when I tried to copy this directory from the other working node the error is gone but now confluence has not started and showing this error 

Unit confluence.service has begun starting up.
Jun 15 22:45:15 conf01 systemd[29749]: confluence.service: Failed to execute command: Permission denied
Jun 15 22:45:15 conf01 systemd[29749]: confluence.service: Failed at step EXEC spawning /opt/atlassian/confluence/bin-- Subject: Process /opt/atlassian/confluence/bin/start-confluence.sh could not be executed
-- Defined-By: systemd

Process /opt/atlassian/confluence/bin/start-confluence.sh could not be executed

[Vikrant Yadav]

@Oday Rafeh  How you fixed this issue? I am also facing the same issue. Kindly suggest. 

[Oday Rafeh]

Hi, @Vikrant Yadav do you still facing this issue? 

[Vikrant Yadav]

Hi @Oday Rafeh  Thanks for the reply. But issue is fixed now. 

[Ilkay Tarik Cansiz]

I am having the same issue with deploying confluence data centre using helm chart, currently confluence is running on open shift cluster and it seems like multicast discovery is not available and we can not create cluster role and cluster role binding since there are other running on the cluster do you have any suggestions?

Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!

at org.springframework.orm.hibernate.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:473)

at com.atlassian.confluence.impl.hibernate.ConfluenceHibernateTransactionManager.doBegin(ConfluenceHibernateTransactionManager.java:41)

at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)

at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)

at jdk.internal.reflect.GeneratedMethodAccessor105.invoke(Unknown Source)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:566)

at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)

at com.sun.proxy.$Proxy79.getTransaction(Unknown Source)

at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:595)

at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:382)

at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)

at com.sun.proxy.$Proxy87.retrieve(Unknown Source)

at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)

at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:24)

at com.atlassian.confluence.impl.settings.BandanaGlobalSettingsManager.getGlobalSettings(BandanaGlobalSettingsManager.java:43)

at com.atlassian.confluence.impl.settings.DelegatingLegacySettingsManager.getGlobalSettings(DelegatingLegacySettingsManager.java:57)

at com.atlassian.confluence.status.service.DefaultSystemInformationService.isShowInfoOn500(DefaultSystemInformationService.java:326)

at org.apache.jsp._500page_jsp._jspService(_500page_jsp.java:406)

at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)

at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)

... 66 more

Caused by: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!

at com.hazelcast.instance.HazelcastInstanceProxy.getOriginal(HazelcastInstanceProxy.java:321)

at com.hazelcast.instance.HazelcastInstanceProxy.getCluster(HazelcastInstanceProxy.java:219)

at jdk.internal.reflect.GeneratedMethodAccessor172.invoke(Unknown Source)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:566)

at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)

at com.sun.proxy.$Proxy73.getCluster(Unknown Source)

at com.hazelcast.hibernate.HazelcastTimestamper.nextTimestamp(HazelcastTimestamper.java:38)

at com.hazelcast.hibernate.AbstractHazelcastCacheRegionFactory.nextTimestamp(AbstractHazelcastCacheRegionFactory.java:76)

at org.hibernate.internal.AbstractSharedSessionContract.<init>(AbstractSharedSessionContract.java:142)

at org.hibernate.internal.AbstractSessionImpl.<init>(AbstractSessionImpl.java:29)

at org.hibernate.internal.SessionImpl.<init>(SessionImpl.java:253)

at org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl.openSession(SessionFactoryImpl.java:1229)

at org.hibernate.internal.SessionFactoryImpl.openSession(SessionFactoryImpl.java:449)

at com.atlassian.confluence.impl.core.persistence.hibernate.SwitchableCachingSessionFactory.openSession(SwitchableCachingSessionFactory.java:117)

at jdk.internal.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:566)

at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)

at com.sun.proxy.$Proxy80.openSession(Unknown Source)

at com.atlassian.hibernate.adapter.bridge.factory.SessionFactoryV2orV5BridgeProxy.openSession(SessionFactoryV2orV5BridgeProxy.java:95)

at org.springframework.orm.hibernate.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:413)