It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Ehcache replication exceptions in JIRA DataCenter

Hi,

I've setup an JIRA Datacenter (based on 2 JIRA Software 7.2.0) like shown here https://confluence.atlassian.com/adminjiraserver071/installing-jira-data-center-802592197.html

I'm seeing a lot of exceptions in the log regarding ehcache replication.

Both nodes are active and responding. Load balancer (Apache2) can access and properly proxy to both instances. All configs are installation default - I've changed only $JIRA_INSTALL/conf/server.xml (added jvmRoute and HTTPS handling).

Ports open on both machines: 8080 (JIRA), 8000 (debug), 40001 (ehcache).

How can I fix this?

Exception below:

2017-02-23 10:14:44,397 myWorkHostChecker:thread-1 ERROR      [n.s.ehcache.distribution.RMISynchronousCacheReplicator] Exception on replication of putNotification. RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled). Continuing...
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:355)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at net.sf.ehcache.distribution.RMICachePeer_Stub.put(Unknown Source)
        at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:149)
        at com.atlassian.cache.ehcache.replication.rmi.RMISynchronousCacheReplicator.replicateViaCopy(RMISynchronousCacheReplicator.java:60)
        at com.atlassian.cache.ehcache.replication.rmi.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:48)
        at net.sf.ehcache.event.RegisteredEventListeners.internalNotifyElementPut(RegisteredEventListeners.java:192)
        at net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:170)
        at net.sf.ehcache.Cache.notifyPutInternalListeners(Cache.java:1648)
        at net.sf.ehcache.Cache.putInternal(Cache.java:1618)
        at net.sf.ehcache.Cache.put(Cache.java:1543)
        at net.sf.ehcache.Cache.put(Cache.java:1508)
        at com.atlassian.cache.ehcache.DelegatingCache.put(DelegatingCache.java:82)
        at com.atlassian.mywork.client.service.HostIdCache.setHost(HostIdCache.java:47)
        at com.atlassian.mywork.client.service.HostServiceImpl.setRegisteredHost(HostServiceImpl.java:211)
        at com.atlassian.mywork.client.service.HostServiceImpl.updateHostAvailability(HostServiceImpl.java:200)
        at com.atlassian.mywork.client.service.HostServiceImpl.enable(HostServiceImpl.java:102)
        at com.atlassian.mywork.client.service.ServiceSelectorImpl$2.run(ServiceSelectorImpl.java:194)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:315)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        ... 3 more
Caused by: java.lang.ClassNotFoundException: com.atlassian.mywork.client.service.HostServiceImpl$HostType (no security manager: RMI class loader disabled)
        at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:396)
        at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:186)
        at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
        at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
        at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:219)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
        at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1733)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
        at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:503)
        at net.sf.ehcache.Element.readObject(Element.java:884)
        ... 2 filtered
        at java.lang.reflect.Method.invoke(Method.java:498)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
        at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:326)
        at sun.rmi.server.UnicastServerRef.unmarshalParametersUnchecked(UnicastServerRef.java:592)
        at sun.rmi.server.UnicastServerRef.unmarshalParameters(UnicastServerRef.java:580)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:310)
        ... 12 more

2 answers

2 votes
Chris Fuller Atlassian Team Feb 23, 2017

That looks to me like the MyWork plugin is attempting to use its own class as a cache key.  This does not work, because cache replication occurs over RMI and, as the exception indicates, the RMI class loader restricts which classes it will load for security reasons (and plugin classes are not available to it).  This should be considered a bug in the MyWork plugin, which is owned internally within Atlassian by Confluence's development team.

Please report this to support.

0 votes

There is a bug record opened to address this matter which was fixed on JIRA Sever 7.2.6, please refer: https://jira.atlassian.com/browse/JRA-59979.

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Jira

Managing Incidents - The roof is on FIRE!

The roof is on FIRE… network outages, broken processes, upset clients and employees. Each day seemed to bring more and more issues. Incidents were communicated via email, messengers (skype or teams) ...

128 views 0 1
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you