Ehcache replication exceptions in JIRA DataCenter

Hi,

I've setup an JIRA Datacenter (based on 2 JIRA Software 7.2.0) like shown here https://confluence.atlassian.com/adminjiraserver071/installing-jira-data-center-802592197.html

I'm seeing a lot of exceptions in the log regarding ehcache replication.

Both nodes are active and responding. Load balancer (Apache2) can access and properly proxy to both instances. All configs are installation default - I've changed only $JIRA_INSTALL/conf/server.xml (added jvmRoute and HTTPS handling).

Ports open on both machines: 8080 (JIRA), 8000 (debug), 40001 (ehcache).

How can I fix this?

Exception below:

2017-02-23 10:14:44,397 myWorkHostChecker:thread-1 ERROR      [n.s.ehcache.distribution.RMISynchronousCacheReplicator] Exception on replication of putNotification. RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled). Continuing...
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:355)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at net.sf.ehcache.distribution.RMICachePeer_Stub.put(Unknown Source)
        at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:149)
        at com.atlassian.cache.ehcache.replication.rmi.RMISynchronousCacheReplicator.replicateViaCopy(RMISynchronousCacheReplicator.java:60)
        at com.atlassian.cache.ehcache.replication.rmi.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:48)
        at net.sf.ehcache.event.RegisteredEventListeners.internalNotifyElementPut(RegisteredEventListeners.java:192)
        at net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:170)
        at net.sf.ehcache.Cache.notifyPutInternalListeners(Cache.java:1648)
        at net.sf.ehcache.Cache.putInternal(Cache.java:1618)
        at net.sf.ehcache.Cache.put(Cache.java:1543)
        at net.sf.ehcache.Cache.put(Cache.java:1508)
        at com.atlassian.cache.ehcache.DelegatingCache.put(DelegatingCache.java:82)
        at com.atlassian.mywork.client.service.HostIdCache.setHost(HostIdCache.java:47)
        at com.atlassian.mywork.client.service.HostServiceImpl.setRegisteredHost(HostServiceImpl.java:211)
        at com.atlassian.mywork.client.service.HostServiceImpl.updateHostAvailability(HostServiceImpl.java:200)
        at com.atlassian.mywork.client.service.HostServiceImpl.enable(HostServiceImpl.java:102)
        at com.atlassian.mywork.client.service.ServiceSelectorImpl$2.run(ServiceSelectorImpl.java:194)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:315)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        ... 3 more
Caused by: java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:396)
        at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:186)
        at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
        at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
        at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:219)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
        at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1733)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
        at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:503)
        at net.sf.ehcache.Element.readObject(Element.java:884)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:498)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
        at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:326)
        at sun.rmi.server.UnicastServerRef.unmarshalParametersUnchecked(UnicastServerRef.java:592)
        at sun.rmi.server.UnicastServerRef.unmarshalParameters(UnicastServerRef.java:580)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:310)
        ... 12 more

2 answers

This widget could not be displayed.
Chris Fuller Atlassian Team Feb 23, 2017

That looks to me like the MyWork plugin is attempting to use its own class as a cache key.  This does not work, because cache replication occurs over RMI and, as the exception indicates, the RMI class loader restricts which classes it will load for security reasons (and plugin classes are not available to it).  This should be considered a bug in the MyWork plugin, which is owned internally within Atlassian by Confluence's development team.

Please report this to support.

This widget could not be displayed.

There is a bug record opened to address this matter which was fixed on JIRA Sever 7.2.6, please refer: https://jira.atlassian.com/browse/JRA-59979.

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Wednesday in New to Jira

Are you planning to trial, or are currently trialling Jira Software? - We want to talk to you!

Hello! I'm Rayen, a product manager at Atlassian. My team and I are working hard to improve the trial experience for Jira Software Cloud. We are interested in   talking to 20 people planning t...

169 views 2 0
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you