Crowd 2.12 doesn't starts after upgrading Java 8 to update 151

[Lokesh Pathak]

After upgrading our crowd 2.12 server to Java 8 update 151 (from 141) it throws tons of exceptions and doesn't come up as usual. Attaching some exceptions below (couldn't find a way to attach entire stack traces). If any additional information is required to triage this issue do let me know.

java.lang.RuntimeException: com.atlassian.plugin.servlet.DownloadException: Unable to stream to the output
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$6.writeTo(ResourceServingHelpers.java:360)
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$2.lambda$writeTo$0(ResourceServingHelpers.java:184)
 at com.atlassian.plugin.cache.filecache.impl.PassThroughCache.cache(PassThroughCache.java:18)
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$2.writeTo(ResourceServingHelpers.java:184)
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$8.writeTo(ResourceServingHelpers.java:384)
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$1.writeTo(ResourceServingHelpers.java:125)
 at com.atlassian.plugin.webresource.impl.http.Controller$6.write(Controller.java:362)
 at com.atlassian.plugin.cache.filecache.impl.PassThroughCache.cache(PassThroughCache.java:18)
 at com.atlassian.plugin.webresource.impl.http.Controller.sendCachedInProduction(Controller.java:359)
 at com.atlassian.plugin.webresource.impl.http.Controller.sendCached(Controller.java:322)
 at com.atlassian.plugin.webresource.impl.http.Controller.serveResources(Controller.java:262)
 at com.atlassian.plugin.webresource.impl.http.Controller.serveBatch(Controller.java:124)
 at com.atlassian.plugin.webresource.impl.http.Router$5.apply(Router.java:78)
 at com.atlassian.plugin.webresource.impl.http.Router$5.apply(Router.java:71)
 at com.atlassian.plugin.webresource.impl.support.http.BaseRouter.callHandler(BaseRouter.java:169)
 at com.atlassian.plugin.webresource.impl.support.http.BaseRouter.dispatch(BaseRouter.java:144)
 at com.atlassian.plugin.webresource.servlet.PluginResourceDownload.serveFile(PluginResourceDownload.java:65)
 at com.atlassian.plugin.servlet.AbstractFileServerServlet.doGet(AbstractFileServerServlet.java:28)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)




Caused by: com.atlassian.plugin.servlet.DownloadException: Unable to stream to the output
 at com.atlassian.plugin.webresource.transformer.TransformerUtils.transformAndStreamResource(TransformerUtils.java:44)
 at com.atlassian.plugin.webresource.transformer.CharSequenceDownloadableResource.streamResource(CharSequenceDownloadableResource.java:42)
 at com.atlassian.plugin.webresource.impl.helpers.ResourceServingHelpers$6.writeTo(ResourceServingHelpers.java:358)
 ... 107 more
Caused by: org.apache.catalina.connector.ClientAbortException: java.net.SocketException: Connection reset
 at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:367)
 at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:331)
 at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
 at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
 at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
 at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
 at org.apache.commons.io.IOUtils.copy(IOUtils.java:2128)
 at org.apache.commons.io.IOUtils.copy(IOUtils.java:2158)
 at com.atlassian.plugin.webresource.transformer.TransformerUtils.transformAndStreamResource(TransformerUtils.java:40)
 ... 109 more

[Marcin Kempa · Accepted solution]

Hi @Lokesh Pathak!

It looks like you are facing problem described here https://jira.atlassian.com/browse/CWD-4999.

The symptoms and actual root cause seems to be the same as here https://jira.atlassian.com/browse/CWD-4968.

The reason why this is happening now after upgrading Java 8 to update 151 is that the bundled version of zlib changed in Java. There is even a public facing issue fo that https://bugs.openjdk.java.net/browse/JDK-8189789

In this situation, since Crowd 3.1.0 is not yet publicly available, I would suggest disabling gzip compression, see this KB article for more details: https://confluence.atlassian.com/crowdkb/how-to-disable-http-compression-on-crowd-2-9-and-above-856714638.html

Cheers,

Marcin Kempa

[Lokesh Pathak]

Hi @Marcin Kempa 

You are correct and that workaround did work for me. It would be nice to be notified when this issue is fixed so we can get the workaround reverted.

Thanks!

[AnnWorley]

Hi Lokesh,

I am happy you were able to resolve the issue for the time being. The bug has been fixed in the next release of Crowd (3.1) which is not available yet. So it sounds like you would like to be notified when that version comes out.

To get on the mailing list to be informed of new releases, please log into https://my.atlassian.com and choose Email Preferences at the very top.

Thanks,

Ann

[AnnWorley]

We will need to examine your logs more thoroughly to find out why Crowd won't start. I opened a support ticket in your name, you should get a notification from our portal shortly. The support engineer will probably want to start with the materials described in: Provide Crowd Information to Atlassian Support