Issue using HTTPS webhooks

[lauriefdx1]

Hi

I am currently attempting to set up webhooks into Slack so that when a new issue is created a post is made in a Slack channel.

Using webhook testing sites, I can see that the data is received when HTTP is used, however when swapping to HTTPS no data is received.

I can see HTTPS traffic in tcpdump to/from jira while the webhook is being posted but no data is received.

No error messages are to be found in the jira log files.

Version: v7.5.2#75007-sha1:9f5725b

[lauriefdx1]

I've finally had a chance to look at this again with fresh eyes. It turns out that the cacerts keystore that was installed by debian package ca-certificates-java was empty., even after deleting the package and reinstalling.

Copying this file from another machine solved the issue.

[Daniel Eads _unmonitored account_]

Hi Laurie,

We had a bear of a time trying to get Slack to respond properly to webhooks. Eventually we just went with a plugin. It's not as easy to use as the HipChat connector, but it is what it is. Using a plugin helps avoid having to make changes on workflows each time you want to add an alert. https://marketplace.atlassian.com/plugins/com.jsoft.jira.plugins.jira-slack/server/overview

I realize you're on Server, but any Cloud folks coming through reading this answer might benefit from these Cloud plugins (free!):

https://marketplace.atlassian.com/plugins/com.atlassian.jira.slack/cloud/overview

https://marketplace.atlassian.com/plugins/jiraslackintegration/cloud/overview

Cheers,
Daniel

[lauriefdx1]

Hi Daniel

This isn't just an issue with Slack, but all webhooks which post to HTTPS. 

For example, when I use the testing site https://webhook.site, posting via HTTP works and I can see the Jira JSON data on that site, however when posting via HTTPS nothing seems to be received by webhook.site.

Further, after enabling DEBUG for webhooks, I can successfully post to Slack via HTTPS the JSON which is output in the log file using something like postman or curl.

[Daniel Eads _unmonitored account_]

Oh interesting. I don't know about Slack, but webhook.site uses Let's Encrypt, which isn't supported in Java's trust store until 8u101. If you're using a system-installed JRE instead of the one that comes bundled with JIRA, that might be causing a trust issue. In your System Info page in JIRA, what do you have under Java Version?

[lauriefdx1]

Java Version	1.8.0_151
Java Vendor	Oracle Corporation
JVM Version	1.8
JVM Vendor	Oracle Corporation
JVM Implementation Version	25.151-b12
Java Runtime	OpenJDK Runtime Environment
Java VM	OpenJDK 64-Bit Server VM

Not sure why Oracle is mentioned here when its OpenJDK installed. I saw that slack's webhooks domain uses GeoTrust - could this be something to do with Google's proposal to revoke GeoTrust certs?

Is there any way to get an error from the webhook logs? Nothing is output under DEBUG other than the data that it has attempted to post?

[Daniel Eads _unmonitored account_]

When I went to get more info about webhook logging, I came across this article about doing it. Maybe you could add com.atlassian.webhooks as a logger as described in the article, then set the level to TRACE to see if you can get any more info?

Another thought I'm wondering about is if you have a reverse proxy in front of your Jira. That article mentions trying to reproduce the problem directly from Jira without a web server in front (Apache, nginx, HAProxy, F5, etc) - maybe something between the proxy configuration with Jira or the proxy itself might be stopping the requests from terminating correctly.

[lauriefdx1]

Hi Daniel
I have set com.atlassian.webhooks to TRACE already but all it outputs is the JSON which is being submitted. There are no errors or success/fail messages at all, this is all I can see:
2017-11-15 23:25:07,944 Web-Hook-Publisher-0 DEBUG (username redacted) 183x70x1 10mv6vm 10.1.1.254 /secure/CreateIssueDetails.jspa [c.a.webhooks.plugin.PublishTaskFactoryImpl$PublishTaskImpl] Posting to web hook at '(url)', body is: (json data redacted)

We don't currently have a reverse proxy set up - its a direct HTTPS connection to the JIRA server.

[Daniel Eads _unmonitored account_]

Got to admit I'm stumped then! We have webhooks on our Jira instance going to Zapier over HTTPS that are working fine. We're on Jira 7.5.1, hard to imagine a single bugfix release would be responsible for that kind of problem. I still think it has something to do with your environment but am not sure what to suggest checking next.