Method Not Allowed when attempting to create application link between JIRA and Confluence

[Dan Foster]

Hey Folks,

I've been scratching my head trying to figure this out for a few days but to no avail.  We're in the process of moving both our JIRA and Confluence instances behind IIS8 with SSL and while we seem to have everything else working, we can't get the Application Links to work between the two applications.

The background:

• We're using JIRA 7.1.4
• We're using Confluence 5.9.8
• We're using IIS8 on Windows 2012
• This is a test environment that has undergone a restore from our fully working production system.
• We were unable to remove the application links on our test environment and they had to be removed manually through some DB hacking following this article: https://confluence.atlassian.com/jirakb/how-to-remove-application-link-directly-through-jira-database-285839519.html
• SSL Certs have been provided by a corporate CA and the CA has been added to the cacerts file for both JIRA and Confluence (this actually fixed our plugin problem during setup as documented here: https://confluence.atlassian.com/jirakb/how-to-fix-gadget-titles-showing-as-__msg_gadget-813697086.html)
• While our production environment has separate servers for JIRA and Confluence, our test environment is a single server, however, it has separate IP addresses and hostnames for each application so SNI is not being used.
• Google searching for the latest error (Exception trying to get Applink for manifest with ID bla bla bla) does not reveal anything.  The closest I get is the problem of not finding the manifest file but that part is succeeding as stated in the trace logs (see below).
• We see the same "Method Not Allowed" if we try to initiate the link from either JIRA or Confluence.

Here is the actual trace log I managed to obtain during one of my attempts with the com.atlassian.applinks set to TRACE:

2016-05-25 17:37:48,965 ajp-nio-8009-exec-7 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.cors.rest.CorsFilter] CORS Header [Access-Control-Allow-Credentials] set to [null]
2016-05-25 17:37:48,965 ajp-nio-8009-exec-7 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.cors.rest.CorsFilter] CORS Header [Access-Control-Allow-Credentials] set to [true]
2016-05-25 17:37:48,965 ajp-nio-8009-exec-7 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.c.rest.ui.CreateApplicationLinkUIResource] URL received 'https://confluence-test.mycompany.com/'
2016-05-25 17:37:48,980 ajp-nio-8009-exec-3 DEBUG myuser 1057x1799x2 p37ddf 10.183.34.87 /rest/applinks/3.0/manifest.json [c.a.a.cors.rest.CorsFilter] CORS Header [Access-Control-Allow-Credentials] set to [null]
2016-05-25 17:37:48,980 ajp-nio-8009-exec-3 DEBUG myuser 1057x1799x2 p37ddf 10.183.34.87 /rest/applinks/3.0/manifest.json [c.a.a.cors.rest.CorsFilter] CORS Header [Access-Control-Allow-Credentials] set to [true]
2016-05-25 17:37:49,090 ajp-nio-8009-exec-7 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.core.manifest.AppLinksManifestDownloader] Returning cached manifest for: https://confluence-test.mycompany.com/
2016-05-25 17:37:49,105 ajp-nio-8009-exec-7 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.c.rest.ui.CreateApplicationLinkUIResource] Manifest retrieved successfully
2016-05-25 17:37:49,105 JIRA-EventThread-35 WARN myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.internal.capabilities.DefaultRemoteCapabilitiesService] Exception trying to get Applink for manifest with ID e23c7b27-e2b2-335a-be33-6f63e692ddc8
2016-05-25 17:37:49,105 ajp-nio-8009-exec-7 INFO myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.core.manifest.AppLinksManifestDownloader] Authenticator placeholder.to.ensure.backwards.compatibility specified by remote application e23c7b27-e2b2-335a-be33-6f63e692ddc8 is not installed locally, and will not be used.
2016-05-25 17:37:49,105 ajp-nio-8009-exec-7 INFO myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.core.manifest.AppLinksManifestDownloader] Authenticator placeholder.to.ensure.backwards.compatibility specified by remote application e23c7b27-e2b2-335a-be33-6f63e692ddc8 is not installed locally, and will not be used.
2016-05-25 17:37:49,105 JIRA-EventThread-35 DEBUG myuser 1057x1798x1 p37ddf 10.183.34.87 /rest/applinks/3.0/applicationlinkForm/manifest.json [c.a.a.internal.capabilities.DefaultRemoteCapabilitiesService] Stack trace for manifest with ID e23c7b27-e2b2-335a-be33-6f63e692ddc8
com.atlassian.applinks.internal.exception.NoSuchApplinkException: Application with ID 'e23c7b27-e2b2-335a-be33-6f63e692ddc8' does not exist.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at com.atlassian.applinks.internal.exception.DefaultServiceExceptionFactory.create(DefaultServiceExceptionFactory.java:93)
	at com.atlassian.applinks.internal.exception.DefaultServiceExceptionFactory.create(DefaultServiceExceptionFactory.java:71)
	at com.atlassian.applinks.internal.applink.DefaultApplinkHelper.getApplicationLink(DefaultApplinkHelper.java:38)
	at com.atlassian.applinks.internal.capabilities.DefaultRemoteCapabilitiesService.getApplinkSafe(DefaultRemoteCapabilitiesService.java:298)
	at com.atlassian.applinks.internal.capabilities.DefaultRemoteCapabilitiesService.onManifestDownloaded(DefaultRemoteCapabilitiesService.java:206)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:36)
	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$1$1.run(AsynchronousAbleEventDispatcher.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

Anyone got any ideas?

Thanks,

Dan

[Dan Foster · Accepted solution]

The solution to this was two fold:

1) I ended up doing a production restore to our test environment (DB, Application Data directory, and Application directory) and after doing all that I managed to get Application Links to function between the two applications using direct (non-IIS) connections.  However, trying to setup the link to go through IIS still showed the same error in the question above so the troubleshooting continued.

2) After getting the app links to work between JIRA and Confluence directly, I managed to finally stumble on a Greenhopper 405 error behind IIS problem listed here:  https://answers.atlassian.com/questions/68533.  I used the solution provided which was to disable WebDAV for the IIS sites for JIRA and Confluence (as it's enabled at the server level).   This is done by adding the following block to the web.config file for the sites in question and restarting the server:

<modules>
    <remove name="WebDAVModule"/>
</modules>
<handlers accessPolicy="Read, Execute, Script">
    <remove name="WebDAV" />
</handlers>

Now the application links work exactly as expected and take the expected 30 seconds to setup.

[Jonas Andersson]

Just a few things to check. Make sure gzip compression of http headers are disabled in both JIRA and confluence. Unrelated to the errors, also make sure alt_names (short and FQDN) are all in the certs. A last workaround is to bypass SSL all together on the host and instead set up the applicationlinks between localhost:8060 and localhost:8080 (defaults, so might not be corresponding on your apps).

[Nic Brough -Adaptavist-]

I've seen this before on IIS - in my case, it was IIS silently dropping the headers due to some silly config setting, but I think Jonas's answer covers all the usual culprits, so you definitely need to check all of that before you do anything else.

But check the connector settings in Tomcat's server.xml too - I've recently had very similar error messages in Apache caused by the connectors being wrong.

[Dan Foster]

@Jonas Andersson, thanks for the suggestions.  Here's the results:

• GZIP compression is turned off in both JIRA and Confluence
• I've tried using the direct localhost entires with the correct ports (8080 and 8090) without success (same error).
• I don't have a lot of control over the alt_names but bypassing SSL didn't work either so...

@Nic Brough [Adaptavist], I don't suppose you remember what your config was or what headers were being dropped?  Although bypassing IIS altogether with the localhost suggestion didn't work either so I don't think this is an SSL/IIS problem.  I'm leaning towards it being an Application Link issue since that subsystem has been a running problem since the beginning.

[Nic Brough -Adaptavist-]

Ah, no, it doesn't matter - if it's failing on localhost, you need to fix that before worrying about IIS.

(But if you insist, on IIS, the "security guru" had decided to do some sort of pointless filtering on outgoing headings.  Once we'd got rid of him it was all good.  On the Tomcat side, I'd got the "scheme" wrong in the connector - needed to be https as I was serving it it up on an encrypted connection, even though the tomcat was http)