It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

HTTP Status 500 - Cannot call sendError() after the response has been committed

I've a confluence cluster env. with 2 nodes.

Everything works fine when I start only the first node, but when I start the second one, I'm getting this error:

HTTP Status 500 - com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE

com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!

I've already increased the JAVA_Memory.

FULL LOG:

type Exception report

message com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE

description The server encountered an internal error that prevented it from fulfilling this request.

exception

java.util.concurrent.CompletionException: com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
root cause

com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE
com.atlassian.vcache.internal.legacy.LegacyUtils.mapException(LegacyUtils.java:51)
com.atlassian.vcache.internal.legacy.LegacyStableReadExternalCache.mapException(LegacyStableReadExternalCache.java:125)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:93)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:69)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.put(AbstractStableReadExternalCache.java:239)
com.atlassian.vcache.internal.core.metrics.TimedExternalWriteOperationsUnbuffered.put(TimedExternalWriteOperationsUnbuffered.java:36)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
root cause

com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
com.hazelcast.spi.AbstractDistributedObject.throwNotActiveException(AbstractDistributedObject.java:104)
com.hazelcast.spi.AbstractDistributedObject.lifecycleCheck(AbstractDistributedObject.java:99)
com.hazelcast.spi.AbstractDistributedObject.getNodeEngine(AbstractDistributedObject.java:93)
com.hazelcast.spi.AbstractDistributedObject.toData(AbstractDistributedObject.java:67)
com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:126)
com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:117)
com.atlassian.cache.hazelcast.HazelcastCache.put(HazelcastCache.java:115)
com.atlassian.confluence.cache.DefaultConfluenceCache.put(DefaultConfluenceCache.java:58)
com.atlassian.confluence.cache.ConfluenceMonitoringCache.put(ConfluenceMonitoringCache.java:87)
com.atlassian.vcache.internal.legacy.LegacyUtils.directPut(LegacyUtils.java:38)
com.atlassian.vcache.internal.legacy.LegacyStableReadExternalCache.internalPut(LegacyStableReadExternalCache.java:64)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.lambda$null$17(AbstractStableReadExternalCache.java:242)
com.atlassian.vcache.internal.core.service.VCacheLock.withLock(VCacheLock.java:33)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.lambda$put$18(AbstractStableReadExternalCache.java:242)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:83)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:69)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.put(AbstractStableReadExternalCache.java:239)
com.atlassian.vcache.internal.core.metrics.TimedExternalWriteOperationsUnbuffered.put(TimedExternalWriteOperationsUnbuffered.java:36)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
note The full stack trace of the root cause is available in the Apache Tomcat/8.0.50 logs.

1 answer

0 votes

Hello!

Could you tell us, is this a fresh install of Confluence Data Center, or did you originally have a Confluence on a single server deployment that you are trying to migrate to Data Center?

Regards,

Shannon

Hi Shannon, thanks for answer me.

Yes, this is a fresh install. What I'm doing is:

1- setup the first node;

2- start the first node;

3- setup all settings(cluster name, db, etc)

4-Shutdown it down;

5- Copy to the second node;

6- Start the first node;

7- Start the second node.

Obs.: Everything is being building in AWS.

Ann Worley Atlassian Team Mar 26, 2018

Hi Lessandro,

 

Please check on the host VM of the second node: can it ping to the IP of the first node and reach it? (If not, please ensure that both nodes are in the same security group and can "talk" to each other)

If you are using quickformation templates, please check your setup against the following guide: Running Confluence Data Center in AWS

Thanks,

Ann

Hi Ann, 

Yes, both can "talk" to each other.

I'm not using quickformation. 

Ted Costales Atlassian Team Mar 26, 2018

Hi @Lessandro:

If this is your first time setting up Confluence Data Center in AWS we would recommend using the QuickFormation templates that @Ann Worley had linked. However, it looks like you're attempting to configure this manually.

Based on the errors you posted in your original comment, my guess is that both nodes are able to talk to the database but not each other on the common ports that Confluence uses. This can cause a Cluster Panic event, which forces nodes in the cluster to terminate if they observe database writes from a third party source.

How are you verifying that both nodes are "talking" to each other? Are you using ping, netstat, some other kind of configuration? In AWS, are both nodes in the same target group with traffic routing through an ELB? What is your cluster join type: unicast, multicast, or AWS IAM?

There are a lot of unknowns in this thread. If this is your first time installing Data Center, try using this guide and following it step by step:

Installing Confluence Data Center

If you are running into issues following these steps exactly, then let us know where in the steps you're having troubles so we can analyze further.

Thanks Ted!

I recreated the environment from scratch and now it is working. I believe my error was because I was setting the shared directory as home directory.

Anyway, thanks again. 

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Confluence Cloud

What's New in Confluence Cloud – October 2019 Edition

Hey there! When you can pack so much information onto a Confluence page, it can be challenging to whittle down what’s most important. Here are a few of our favorite hacks for finding need-to-know i...

63,698 views 63 153
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you