It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

What Jira operations cause synchronous activity in RMISynchronousCacheReplicator?

We suffered a brief network partition in our JIRA Datacenter setup. Among other exceptions, we noticed the usage of RMISynchronousCacheReplicator in some stacktraces in http threads. When do user facing threads cause synchronous activity? Can this be configured to be asynchronous?

Stacktrace from our incident: Connection timed out
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(
        at sun.rmi.transport.tcp.TCPChannel.createConnection(
        at sun.rmi.transport.tcp.TCPChannel.newConnection(
        at sun.rmi.server.UnicastRef.invoke(
        at net.sf.ehcache.distribution.RMICachePeer_Stub.remove(Unknown Source)
        at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicateRemovalNotification(
        at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.notifyElementRemoved(
        at net.sf.ehcache.event.RegisteredEventListeners.internalNotifyElementRemoved(
        at net.sf.ehcache.event.RegisteredEventListeners.notifyElementRemoved(
        at net.sf.ehcache.Cache.notifyRemoveInternalListeners(
        at net.sf.ehcache.Cache.removeInternal(
        at net.sf.ehcache.Cache.remove(
        at net.sf.ehcache.Cache.remove(
        at net.sf.ehcache.constructs.EhcacheDecoratorAdapter.remove(
        at com.atlassian.cache.ehcache.LoadingCache.remove(
        at com.atlassian.cache.ehcache.DelegatingCache.remove(
        at com.atlassian.jira.avatar.CachingTaggingAvatarStore.getByIdTagged(
        at com.atlassian.jira.avatar.AvatarManagerImpl.getByIdTagged(
        at com.atlassian.jira.avatar.AvatarManagerImpl.lambda$transformToJIRAAvatar$480(
        at com.atlassian.fugue.Option$Some.fold(
        at com.atlassian.fugue.Option.flatMap(
        at com.atlassian.jira.avatar.AvatarManagerImpl.transformToJIRAAvatar(
        at com.atlassian.jira.avatar.AvatarManagerImpl.processAvatarData(
        at com.atlassian.jira.avatar.AvatarManagerImpl.readAvatarData(
        at com.atlassian.jira.web.servlet.AvatarToStream.sendAvatar(
        at com.atlassian.jira.web.servlet.AbstractAvatarServlet.defaultDoGet(
        at com.atlassian.jira.web.servlet.ViewUserAvatarServlet.defaultDoGet(
        at com.atlassian.jira.web.servlet.AbstractAvatarServlet.doGet(
        at javax.servlet.http.HttpServlet.service(
        at javax.servlet.http.HttpServlet.service(

1 answer

1 accepted

0 votes
Answer accepted
crf Atlassian Team Oct 24, 2016

When do user facing threads cause synchronous activity?

Essentially, any write operation to a synchronous cache or the initial access to a replicated cache that has not yet bootstrapped values from another node.  At one point in time the Users and Groups caches behaved this way, but I think that was fixed in 7.0.  I have 7.1.0 checked out and can tell by the code that is definitely fixed there, so the only cache I know would do this is the TaskManager's.

Can this be configured to be asynchronous?

It definitely can in the code:

I could be wrong, but my understanding is that we use asynchronous replication by default and that only caches that explicitly opt in to being synchronous will replicate in that manner.  The only such cache I'm aware of offhand is the one for the TaskManager, which is one of the few value-replicating caches in JIRA (most replicate invalidation only).

You should be able to tell from a thread dump which cache was doing synchronous replication.  Without knowing which caches were involved, it isn't really possible to fairly evaluate what the potential consequences of changing them from synchronous to asynchronous might be.  It could be that the cache's author was just being paranoid, or it could be that asynchronous replication destabilized behaviour for their code, so they specified synchronous replication as a workaround.


thanks. indeed it looked to be from a removal from a cache related to avatars.

crf Atlassian Team Oct 25, 2016

Hrmmmm... That isn't a cache I would expect to need to be synchronous, so I may be mistaken about the default?  I don't work on JDC anymore, so this is mostly from memory.  I do know that avatars were moved from the core to a plugin, so there may be something that changed during that process as well.  I'd probably need a stack trace to figure out whether it's peculiar to the avatar cache or not.

i added the full stacktrace.

crf Atlassian Team Oct 25, 2016

From the stack trace a bit of digging, my guesses are:

  1. I don't see any custom settings on this cache, so we seem to be using synchronous replication by default.  There may be an opportunity to improve performance here by making cache replication asynchronous instead, but that could also cause instability if parts of the system don't respond as well to eventual consistency semantics, so it would need a lot of testing.
  2. This particular cache is doing a removal (and thus a replicated event) on every read request, which is just plain dumb.  You should report this to support.  The nasty code is this (from CachingTaggingAvatarStore):

public Avatar getByIdTagged(final Long avatarId) {
    try {
        return taggedAvatars.get(avatarId, () -> tagAndRetrieve(avatarId)).orElse(null);
    } finally {


That `remove` probably isn't necessary at all.  The point is that once an avatar is "tagged" with the metadata that confirms it came from JIRA, we don't need to hold onto the untagged version of it anymore.  But on a cache hit, that untagged one is already gone, so the remove on the untagged avatar is just wasteful.  It could be moved inside the loading function or possibly removed altogether, depending on what the surrounding code does (I didn't dig in further than that).


crf Atlassian Team Oct 25, 2016

In short, I think you saw these stack up not because they are synchronous, but because they are unnecessarily and excessively common replications.

can you suggest the right support address for this?

crf Atlassian Team Oct 25, 2016

I think is the correct place.  (And if it isn't, they'll know what is and forward it on).

thanks so much.

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted in Jira

How InVision centralized their tools and scaled their remote team with Atlassian and Slack

Hi Atlassian Community, We recently published a case study that we thought you might be interested in. Learn about how InVision built their fully remote company’s culture using Atlassian and Slack ...

247 views 0 1
Join discussion

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you