It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

HTTP Status 500 - Cannot call sendError() after the response has been committed

Lessandro Mar 22, 2018

I've a confluence cluster env. with 2 nodes.

Everything works fine when I start only the first node, but when I start the second one, I'm getting this error:

HTTP Status 500 - com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE

com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!

I've already increased the JAVA_Memory.

FULL LOG:

type Exception report

message com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE

description The server encountered an internal error that prevented it from fulfilling this request.

exception

java.util.concurrent.CompletionException: com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:824)
java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:834)
java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2155)
java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:110)
com.atlassian.confluence.impl.concurrency.CompletionStageUtils.foldResult(CompletionStageUtils.java:19)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
root cause

com.atlassian.vcache.ExternalCacheException: Failed due to UNCLASSIFIED_FAILURE
com.atlassian.vcache.internal.legacy.LegacyUtils.mapException(LegacyUtils.java:51)
com.atlassian.vcache.internal.legacy.LegacyStableReadExternalCache.mapException(LegacyStableReadExternalCache.java:125)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:93)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:69)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.put(AbstractStableReadExternalCache.java:239)
com.atlassian.vcache.internal.core.metrics.TimedExternalWriteOperationsUnbuffered.put(TimedExternalWriteOperationsUnbuffered.java:36)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
root cause

com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
com.hazelcast.spi.AbstractDistributedObject.throwNotActiveException(AbstractDistributedObject.java:104)
com.hazelcast.spi.AbstractDistributedObject.lifecycleCheck(AbstractDistributedObject.java:99)
com.hazelcast.spi.AbstractDistributedObject.getNodeEngine(AbstractDistributedObject.java:93)
com.hazelcast.spi.AbstractDistributedObject.toData(AbstractDistributedObject.java:67)
com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:126)
com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:117)
com.atlassian.cache.hazelcast.HazelcastCache.put(HazelcastCache.java:115)
com.atlassian.confluence.cache.DefaultConfluenceCache.put(DefaultConfluenceCache.java:58)
com.atlassian.confluence.cache.ConfluenceMonitoringCache.put(ConfluenceMonitoringCache.java:87)
com.atlassian.vcache.internal.legacy.LegacyUtils.directPut(LegacyUtils.java:38)
com.atlassian.vcache.internal.legacy.LegacyStableReadExternalCache.internalPut(LegacyStableReadExternalCache.java:64)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.lambda$null$17(AbstractStableReadExternalCache.java:242)
com.atlassian.vcache.internal.core.service.VCacheLock.withLock(VCacheLock.java:33)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.lambda$put$18(AbstractStableReadExternalCache.java:242)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:83)
com.atlassian.vcache.internal.core.service.AbstractExternalCache.perform(AbstractExternalCache.java:69)
com.atlassian.vcache.internal.core.service.AbstractStableReadExternalCache.put(AbstractStableReadExternalCache.java:239)
com.atlassian.vcache.internal.core.metrics.TimedExternalWriteOperationsUnbuffered.put(TimedExternalWriteOperationsUnbuffered.java:36)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:184)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:168)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:151)
com.atlassian.confluence.impl.vcache.SynchronousExternalCache.put(SynchronousExternalCache.java:138)
com.atlassian.confluence.languages.DefaultLocaleManager.setRequestLanguages(DefaultLocaleManager.java:213)
com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:45)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.lambda$doFilter$3(VCacheRequestContextFilter.java:44)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:87)
com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:71)
com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:43)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:65)
com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)
com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
com.atlassian.confluence.impl.servlet.HoldingUntilStartedFilter.doFilter(HoldingUntilStartedFilter.java:88)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
note The full stack trace of the root cause is available in the Apache Tomcat/8.0.50 logs.

1 answer

0 votes
Shannon Spaniol Atlassian Team Mar 23, 2018

Hello!

Could you tell us, is this a fresh install of Confluence Data Center, or did you originally have a Confluence on a single server deployment that you are trying to migrate to Data Center?

Regards,

Shannon

Lessandro Mar 23, 2018

Hi Shannon, thanks for answer me.

Yes, this is a fresh install. What I'm doing is:

1- setup the first node;

2- start the first node;

3- setup all settings(cluster name, db, etc)

4-Shutdown it down;

5- Copy to the second node;

6- Start the first node;

7- Start the second node.

Obs.: Everything is being building in AWS.

Ann Worley Atlassian Team Mar 26, 2018

Hi Lessandro,

 

Please check on the host VM of the second node: can it ping to the IP of the first node and reach it? (If not, please ensure that both nodes are in the same security group and can "talk" to each other)

If you are using quickformation templates, please check your setup against the following guide: Running Confluence Data Center in AWS

Thanks,

Ann

Lessandro Mar 26, 2018

Hi Ann, 

Yes, both can "talk" to each other.

I'm not using quickformation. 

Ted Costales Atlassian Team Mar 26, 2018

Hi @Lessandro:

If this is your first time setting up Confluence Data Center in AWS we would recommend using the QuickFormation templates that @Ann Worley had linked. However, it looks like you're attempting to configure this manually.

Based on the errors you posted in your original comment, my guess is that both nodes are able to talk to the database but not each other on the common ports that Confluence uses. This can cause a Cluster Panic event, which forces nodes in the cluster to terminate if they observe database writes from a third party source.

How are you verifying that both nodes are "talking" to each other? Are you using ping, netstat, some other kind of configuration? In AWS, are both nodes in the same target group with traffic routing through an ELB? What is your cluster join type: unicast, multicast, or AWS IAM?

There are a lot of unknowns in this thread. If this is your first time installing Data Center, try using this guide and following it step by step:

Installing Confluence Data Center

If you are running into issues following these steps exactly, then let us know where in the steps you're having troubles so we can analyze further.

Lessandro Apr 04, 2018

Thanks Ted!

I recreated the environment from scratch and now it is working. I believe my error was because I was setting the shared directory as home directory.

Anyway, thanks again. 

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Confluence

6 Awesome Ways to Apply Trello, JIRA and Confluence to your Project

I attended  Atlassian Summit 2019  and learned a lot from the presenters, attendees and knowledgeable Atlassian product managers. The presentations I attended focused on applying Agile, pla...

1,113 views 7 18
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you