Application Health fails even it says 'connected'

Question

Hi,

i have running Confluence 5.9.1 and JIRA 7.4.1 on the same server. There is a running nginx as reverse Proxy (configured as suggested by atlassian docu). Both applications are using the same wildcard certs which is trusted by the applications (imported in the corresponded truststore).
After creating an application link it is added without any problems. In Jira it says 'Connected' and the connection works in both directions.

Now in the health check JIRA says that there is a problem with the application link (it is the only one at the moment).

Debugoutput for 'com.atlassian.applinks':

2017-07-17 17:33:25,470 HealthCheck:thread-5 DEBUG      [c.a.a.core.auth.ApplicationLinkRequestFactoryFactoryImpl] Cannot create request factory with authentication provider 'com.atlassian.applinks.api.auth.types.OAuthAuthenticationProvider' without current user context. 
2017-07-17 17:33:25,472 HealthCheck:thread-5 WARN      [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error while attempting to retrieve status of Application Link 'UUID-UUID-UUID' 
2017-07-17 17:33:25,472 HealthCheck:thread-5 DEBUG      [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error trace for 'UUID-UUID-UUID' 
java.lang.NullPointerException 
        at com.atlassian.applinks.internal.rest.client.RestRequestBuilder.buildRequest(RestRequestBuilder.java:251) 
        at com.atlassian.applinks.internal.rest.client.RestRequestBuilder.build(RestRequestBuilder.java:226) 
        at com.atlassian.applinks.internal.status.oauth.remote.ApplinkAuthenticationOAuthFetchStrategy.fetch(ApplinkAuthenticationOAuthFetchStrategy.java:72) 
        at com.atlassian.applinks.internal.status.oauth.remote.DefaultRemoteOAuthStatusService.fetchInternal(DefaultRemoteOAuthStatusService.java:97) 
        at com.atlassian.applinks.internal.status.oauth.remote.DefaultRemoteOAuthStatusService.fetchOAuthStatus(DefaultRemoteOAuthStatusService.java:84) 
        at com.atlassian.applinks.core.DefaultApplinkStatusService.getApplinkStatus(DefaultApplinkStatusService.java:97) 
        at com.atlassian.support.healthcheck.impl.JiraApplinksStatusService.lambda$null$0(JiraApplinksStatusService.java:52) 
        at com.atlassian.applinks.core.ElevatedPermissionsServiceImpl.executeAs(ElevatedPermissionsServiceImpl.java:31) 
        at com.atlassian.support.healthcheck.impl.JiraApplinksStatusService.lambda$null$3(JiraApplinksStatusService.java:51) 
        at java.util.Optional.map(Optional.java:215) 
        at com.atlassian.support.healthcheck.impl.JiraApplinksStatusService.lambda$getStatus$4(JiraApplinksStatusService.java:45) 
        at java.util.Optional.map(Optional.java:215) 
        at com.atlassian.support.healthcheck.impl.JiraApplinksStatusService.getStatus(JiraApplinksStatusService.java:44) 
        at com.atlassian.support.healthcheck.checks.ApplinksStatusHealthCheck.check(ApplinksStatusHealthCheck.java:33) 
        at com.atlassian.support.healthcheck.impl.PluginSuppliedSupportHealthCheck.check(PluginSuppliedSupportHealthCheck.java:51) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)

Any advice?

Best regards,

Marcel

vboerner · Answer

After upgrading to Jira 7.11 with Confluence 6.10 I see the same message sometimes. I can't reproduce this. Catalina.out contains, e.g.: HealthCheck:thread-4 WARN [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error while attempting to retrieve status of Application Link 'd1e99889-0 0f6-3461-8208-aa2e5c016a66' - OR - HealthCheck:thread-6 WARN [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error while attempting to retrieve status of Application Link 'd1e99889-0 0f6-3461-8208-aa2e5c016a66' The entry occurs every 3 hours next to the ServiceRunner logs The last timestamp was 09:48:55, and ad 09:48:10 I see an entry in the Jira-Webinterface > "Scheduler Details" (unfortunately without a message): CompatibilityPluginScheduler.JobId.hipchatInstallGlancesJob Type RunnableParameters {}Run mode once per clusterSchedule 1 hourLast Run Fri Aug 03 09:48:10 CEST 2018Last run duration 13 millisecondsNext Run Fri Aug 03 10:48:10 CEST 2018Message Deleting the Application-Link on the Jira-site and setting it up again did NOT help. 2018-08-03 13:48:55,802 HealthCheck:thread-6 WARN [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error while attempting to retrieve status of Application Link 'd1e99889-00f6-3461-8208-aa2e5c016a66' 2018-08-03 13:48:55,802 HealthCheck:thread-6 DEBUG [c.a.applinks.core.DefaultApplinkStatusService] Unrecognized error trace for 'd1e99889-00f6-3461-8208-aa2e5c016a66' ... so I searched in the catalina.out for the ApplicationLink-ID from this message ( meaning: "d1e99889..." in my case) There I found another error (shortened): 2018-08-03 03:33:42,425 http-nio-8080-exec-21 WARN anonymous 213x313x1 [......] /rest/applinks/latest/applicationlink/d1e99889-00f6-3461-8208-aa2e5c016a66/authentication/provider ....... Problem encountered authentication OAUTH ........ <<<<<<...I resume that line on the next line: >>>>>>>> ...., error was "consumer_key_unknown", with parameters "{oauth=consumer_key_unknown}" ... that led me to this thread: https://confluence.atlassian.com/kb/oauth-error-oauth_problem-consumer_key_unknown-720406729.html So I looked into the jira database, but there is no table with the name "bandana". I had to look into the confluence-database, and there is indeed a duplicate entry in the "bandana" table: bandanaid ---------- 1638435 1638434 I cant shut down JIRA at the moment, but maybe following that thread could solve the problem

Allison Kellner · Answer

I would be interested to know if there is any additional feedback on the above. We are now seeing similar behavior after upgrading JIRA from 7.1.6 to v7.4.5. We are running on Confluence 6.2.4. We also have Bitbucket, Fisheye/Crucible linked to JIRA as well. We are seeing the Health Check application link error however when checking the application links all are connected with no errors.

I have turned on DEBUG logging to see if the error returns and I can capture what is causing the issue. But if any other details are available prior to then it would be much appreciated!

mharig · Answer

Hey @somethingblue

Do you have any updates/suggestions/advice for me?

Thank you in advanced,

Marcel