Stash application links not working anymore since upgrade to Java 1.8

[Michael T.]

Hi,

Since I've upgraded my server to Java 1.8 (Oracle JRE), application links from Stash to other Atlassian applications are not working anymore. I always get the following exception when trying to add an application link although I've secured Stash with SSL through Apache and not Tomcat, so I'm wondering why an error like this comes up at all:

Caused by: javax.net.ssl.SSLException: java.security.cert.CertificateException: No subject alternative names present
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:287) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:267) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:176) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at com.atlassian.sal.core.net.CustomSSLProtocolSocketFactory.createSocket(CustomSSLProtocolSocketFactory.java:101) ~[sal-core-2.13.4.jar:na]
        at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) ~[commons-httpclient-3.1-atlassian-2.jar:na]
        at com.atlassian.sal.core.net.HttpClientRequest.executeMethod(HttpClientRequest.java:597) ~[sal-core-2.13.4.jar:na]
        at com.atlassian.sal.core.net.HttpClientRequest.executeMethod(HttpClientRequest.java:556) ~[sal-core-2.13.4.jar:na]
        at com.atlassian.sal.core.net.HttpClientRequest.executeAndReturn(HttpClientRequest.java:360) ~[sal-core-2.13.4.jar:na]
        at com.atlassian.applinks.core.manifest.AppLinksManifestDownloader.download1(AppLinksManifestDownloader.java:150) ~[applinks-plugin-4.3.7_1431489370000.jar:na]
        ... 30 common frames omitted
Caused by: java.security.cert.CertificateException: No subject alternative names present
        at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:142) ~[na:1.7.0_79]
        at sun.security.util.HostnameChecker.match(HostnameChecker.java:91) ~[na:1.7.0_79]
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:284) ~[commons-httpclient-3.1-atlassian-2.jar:na]

I'm aware that there is an option to use keytool to specify an alternate server name (the IP in my case) which I'm using with a self-signed certificate, but I do not see why this was not the case with Java 1.7. And do I need the certificate in the key store of the JRE when I've secured Stash with Apache as described in https://confluence.atlassian.com/display/STASH/Securing+Stash+with+Apache+using+SSL ?

Any ideas?

Thanks,

Michael

[ThiagoBomfim · Accepted solution]

Hi Michael,

The solution on this KB should work for you.

Check which JAVA_HOME your Stash instance is using and import the certificate into the right place.

Hope it helps.

Best,
Thiago Bomfim

[Michael T.]

Hi, This is what I did (I added the "-ext san" part because various sources tell that this is the thing to do to get rid of the "subject alternative names present" error): openssl s_client -connect MY_IP_ADDRESS:443 < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > public.crt /usr/lib/jvm/java-8-oracle/jre/bin/keytool -import -alias tomcat -keystore /usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts -file public.crt -ext san=ip:MY_IP_ADDRESS but unfortunately, I still get Caused by: java.security.cert.CertificateException: No subject alternative names present at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:144) ~[na:1.8.0_45] at sun.security.util.HostnameChecker.match(HostnameChecker.java:93) ~[na:1.8.0_45] at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:284) ~[commons-httpclient-3.1-atlassian-2.jar:na] In Stash, I see that this JRE is used: java.home=/usr/lib/jvm/java-8-oracle/jre Any ideas? Thanks

[ThiagoBomfim]

Hi Michael, If what is published here is correct: http://stackoverflow.com/questions/8744607/how-to-add-subject-alernative-name-to-ssl-certs and you did that, did you try restarting Stash? The problem happens because Stash is fetching the certificate from the remote application and that certificate doesn't contain the san entry. Is it a self-signed certificate? If that's the case, you could also try generating the certificate on the remote application: keytool -genkey -alias tomcat -keyalg RSA -ext san=ip:<Remote_App_Address> How are the AppLinks created? Are you using the IP in the AppLinks configuration? Any reason why you're not using the host name defined on the cert? If you use the hostname that should go away.

[Michael T.]

Hi Thiago, Yes, I tried restarting Stash, with no success. It is a self-signed certificate. I was not able to create the AppLinks because after entering the address of the remote application (e.g., Bamboo), I immediately get the following error: "No response was received from the URL you entered - it may not be valid. Please fix the URL below, if needed, and click Continue." Unfortunately I cannot use the host name in the cert, because we do not have a domain for this public server. What do you mean by "generating the certificate on the remote application"? Is that different from what I have done above? Any help is much appreciated.

[ThiagoBomfim]

Hi Michael, Maybe you can't add the AppLinks by entering the address of the remote application because your Stash server can't resolve the name? Have you tried adding an entry to your /etc/hosts for the name (i.e. bamboo.yourdomain.com) and the IP you're using to create the AppLinks? What happens if you try to ping the Bamboo domain name on the server where Stash is running? Checkout the resolution on https://confluence.atlassian.com/display/STASHKB/Stash+Fails+to+Start+Up+with+java.net.UnknownHostException+Exception ?

[Michael T.]

Thanks, but both Bamboo and Stash are running on the same physical server. So there shouldn't be a problem to resolve the remote application. When I try to enter the application link, I'm still getting Caused by: java.security.cert.CertificateException: No subject alternative names present at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:142) ~[na:1.7.0_79] at sun.security.util.HostnameChecker.match(HostnameChecker.java:91) ~[na:1.7.0_79] at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:284) ~[commons-httpclient-3.1-atlassian-2.jar:na] Is there any other chance to diagnose what's missing with the certificates?

[ThiagoBomfim]

Hi Michael, I believe that the best way to look at this would be raising a support issue with us: https://support.atlassian.com That way, we'd be able to have a closer look at what is going on. Cheers.

[Michael T.]

Thanks to Atlassian support and following this resource http://apetec.com/support/GenerateSAN-CSR.htm I was finally able to resolve this issue. Seams that Java 8 really requires certificates with a SAN in that case.

[rrudnicki]

Hi Michael, 

This usually caused when accessing Stash over HTTPS by using the IP address and not the hostname. The No subject alternative names present error indicates that there is no Subject Alternative Name (SAN) contained in the certificate that matches this IP address. Java is very strict about enforcing that the certificate contains a SAN entry for the IP address, if that is how it's being accessed.

On this documentation you will get more information and the resolution:

https://confluence.atlassian.com/display/STASHKB/No+Subject+Alternative+Names+Present+When+Running+the+Backup+Client

Regards, 

Renato Rudnicki

[Michael T.]

Thanks, it is only strange that this wasn't a problem before (with JRE 7). I've never used the SAN in my old certificates. Nevertheless, I've just tried before to recreate the certificate with the keytool like this: /usr/lib/jvm/java-8-oracle/bin/keytool -delete -alias tomcat /usr/lib/jvm/java-8-oracle/bin/keytool -genkey -alias tomcat -keyalg RSA -ext san=ip:MY_IP But still getting the same error. Do I need to do something else? According to https://confluence.atlassian.com/display/STASH/Securing+Stash+with+Apache+using+SSL I actually would only have to refer to certificates from the Apache config. Do you know why I need to use the Java key store and the keytool here at all? Any help is much appreciated.

[Jobin Kuruvilla [Adaptavist]]

When connecting over SSL, you need to import the public certs of other in to the Java keystore. It looks like you lost them after the upgrade to Java.

You can take a look at cacerts file in the old java and find out the certs you had imported.

[Michael T.]

Thanks. Can I just copy /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/security/cacerts to /usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts ?

[Michael T.]

I've just verified with "keytool -list" that I have my certificates in both JRE's. What is the proper way to do everything from scratch? According to https://confluence.atlassian.com/display/STASH/Securing+Stash+with+Apache+using+SSL it is enough to just specify the certificate in the Apache config. Why do I need to import certificates into the Java keystore? I thought this is only a must when I would use Tomcat to secure my Atlassian applications...