Migrating confluence server

[adminuk]

I am migrating confluence DC server along with it's db....and running into this issue. I can not access it over 443 in the browser. I get the following error message:

2024-07-26 10:34:56,887 ERROR [AtlassianEvent::CustomizableThreadFactory-1] [renderer.internal.http.HttpClientFetcher] fetch Unable to perform a request to: https://confluencetest0.test.com/rest/gadgets/1.0/g/messagebundle/en_GB/gadget.common%2Cgadget.confluence javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source) at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(Unknown Source) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(Unknown Source) at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(Unknown Source) at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at com.atlassian.gadgets.renderer.internal.http.WhitelistAwareHttpClient.execute(WhitelistAwareHttpClient.java:54) at com.atlassian.gadgets.renderer.internal.http.ShindigApacheClientAdapter.execute(ShindigApacheClientAdapter.java:42) at com.atlassian.gadgets.renderer.internal.http.HttpClientFetcher.performRequest(HttpClientFetcher.java:122) at com.atlassian.gadgets.renderer.internal.http.HttpClientFetcher.fetch(HttpClientFetcher.java:73) at org.apache.shindig.gadgets.DefaultMessageBundleFactory.fetchBundle(DefaultMessageBundleFactory.java:138) at org.apache.shindig.gadgets.DefaultMessageBundleFactory.getNestedBundle(DefaultMessageBundleFactory.java:111) at org.apache.shindig.gadgets.DefaultMessageBundleFactory.getBundle(DefaultMessageBundleFactory.java:79) at org.apache.shindig.gadgets.variables.VariableSubstituter.substitute(VariableSubstituter.java:47) at com.atlassian.gadgets.renderer.internal.GadgetSpecFactoryImpl.getGadgetSpec(GadgetSpecFactoryImpl.java:127) at com.atlassian.gadgets.renderer.internal.GadgetSpecFactoryImpl.getGadgetSpec(GadgetSpecFactoryImpl.java:83) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137) at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137) at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:241) at jdk.proxy174/jdk.proxy174.$Proxy2461.getGadgetSpec(Unknown Source) at com.atlassian.confluence.plugins.gadgets.metadata.GadgetsMacroMetadataProvider.generateMacroMetadata(GadgetsMacroMetadataProvider.java:188) at com.atlassian.confluence.plugins.gadgets.metadata.GadgetsMacroMetadataProvider.getMacroMetadata(GadgetsMacroMetadataProvider.java:90) at com.atlassian.confluence.plugins.gadgets.metadata.CachingGadgetsMacroMetadataProvider.retrieveMetadataAndUpdateCache(CachingGadgetsMacroMetadataProvider.java:232) at com.atlassian.confluence.plugins.gadgets.metadata.CachingGadgetsMacroMetadataProvider.getData(CachingGadgetsMacroMetadataProvider.java:168) at com.atlassian.confluence.macro.browser.DefaultMacroMetadataManager.buildMacroMetadata(DefaultMacroMetadataManager.java:135) at com.atlassian.confluence.macro.browser.DefaultMacroMetadataManager.onPluginFrameworkStartedEvent(DefaultMacroMetadataManager.java:63) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42) at com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimedListenerInvoker.invoke(ConfluenceListenerHandlersConfiguration.java:96) at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48) at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37) at com.atlassian.confluence.event.ConfluenceEventDispatcher$1.run(ConfluenceEventDispatcher.java:75) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed at java.base/sun.security.validator.PKIXValidator.doValidate(Unknown Source) at java.base/sun.security.validator.PKIXValidator.engineValidate(Unknown Source) at java.base/sun.security.validator.Validator.validate(Unknown Source) at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source) at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source) ... 74 more Caused by: java.security.cert.CertPathValidatorException: validity check failed at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(Unknown Source) at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source) at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source) at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(Unknown Source) at java.base/java.security.cert.CertPathValidator.validate(Unknown Source) ... 79 more Caused by: java.security.cert.CertificateExpiredException: NotAfter: Wed Aug 02 13:29:52 EDT 2023 at java.base/sun.security.x509.CertificateValidity.valid(Unknown Source) at java.base/sun.security.x509.X509CertImpl.checkValidity(Unknown Source) at java.base/sun.security.provider.certpath.BasicChecker.verifyValidity(Unknown Source) at java.base/sun.security.provider.certpath.BasicChecker.check(Unknown Source)

[Nicolas Grossi]

@adminuk It appears that the certified has expired "Caused by: java.security.cert.CertificateExpiredException: NotAfter: Wed Aug 02 13:29:52" ?

[Jim Knepley - ReleaseTEAM]

Good catch... I focused on trust and totally missed that certificated expired in 2023.

[Jim Knepley - ReleaseTEAM]

Hi @adminuk 

The "PKIX path validation failed" error is telling you that the TLS certificate presented by the remote server is not trusted, which is to say that it's not in the JRE trust store.

You have two options: trust the certificate, or install a different certificate that is trusted.

Trust the certificate (not recommended): on the (Linux) server trying to connect, as root:

echo | openssl s_client -connect confluencetest0.test.com 2>/dev/null | openssl x509 > /etc/pki/ca-trust/source/anchors/confluencetest0.pem

/usr/bin/update-ca-trust

/opt/atlassian/confluence/jre/bin/keytool \\
    -import -trustcacerts -cacerts -storepass changeit -noprompt -alias confluencetest0 \\
    -file "/etc/pki/ca-trust/source/anchors/confluencetest0.pem"

This is not the ideal solution because it trusts the "end node" certificate which will eventually expire and become invalid, and you're back where you started. I show it here because it's a "quick and dirty" (and more importantly, free) solution if the server is being decommissioned soon, etc.

Install a different certificate: get a certificate issued by a trusted CA. You can get one for free from Let's Encrypt (if your IT/security team allows it) or a commercial CA like Sectigo or DigiCert.

See the Atlassian documentation (option 2) for instructions.