Jira Server ERR_SPDY_PROTOCOL_ERROR with nginx proxy

[bigwhale]

After an upgrade to 8.40 I had to renew my license and I stumbled upon an error on the license renew page (/secure/ConfirmNewInstallationWithOldLicense!default.jspa)

Static asset files failed to load (meta-all.js, jquery-min.js, metal-all.css) with a status (failed) in Chrome/Brave. In Firefox HTTP status was still 200, but Firefox said that files failed to load.

These are the relevant sections of the config files:

 <Connector port="8080"
            relaxedPathChars="[]|"
            relaxedQueryChars="[]|{}^&#x5c;&#x60;&quot;&lt;&gt;"
            maxThreads="150"
            minSpareThreads="25"
            connectionTimeout="20000"
            enableLookups="false"
            maxHttpHeaderSize="8192"
            protocol="HTTP/1.1"
            useBodyEncodingForURI="true"
            redirectPort="8443"
            acceptCount="100"
            disableUploadTimeout="true"
            bindOnInit="false"
            secure="true" scheme="https"
            proxyName="jira.lubica.net"
            proxyPort="443"/>

upstream jira {
  server 127.0.0.1:8080 fail_timeout=0;
}

server {
  listen 80;
  server_name jira.lubica.net;

  location / {
    proxy_set_header Host $host:$server_port;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass http://jira;
    client_max_body_size 10M;
  }

  access_log /var/log/nginx/jira/access.log;
  error_log /var/log/nginx/jira/error.log;

  listen 443 ssl; # managed by Certbot
  ssl_certificate /etc/letsencrypt/live/jira.lubica.net/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/jira.lubica.net/privkey.pem; # managed by Certbot
  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

  resolver 8.8.8.8;
  ssl_stapling on;
  ssl_session_cache shared:SSL:10m;
  ssl_trusted_certificate /etc/letsencrypt/live/jira.lubica.net/fullchain.pem;

  ssl_dhparam /etc/nginx/dhparam.pem;

  if ($scheme != "https") {
    return 301 https://$host$request_uri;
  } # managed by Certbot
}

If I access the files directly, nginx will serve them without any problems. I get these errors only when files are loaded subsequently when accessing ConfirmNewInstallationWithOldLicense!default.jspa

Nginx logs an error about prematurely closed connection:

2019/09/19 10:39:04 [error] 19220#19220: *7 upstream prematurely closed connection while reading upstream, client: xxx.xxx.xxx.xxx, server: jira.lubica.net, request: "GET /static-assets/jquery-min.js HTTP/2.0", upstream: "http://127.0.0.1:8080/static-assets/jquery-min.js", host: "jira.lubica.net", referrer: "https://jira.lubica.net/secure/ConfirmNewInstallationWithOldLicense!default.jspa"

 I tried various different configurations and settings and nothing really works and I can't even pinpoint the problem, if it is with nginx configuration or with Jira.

I am running nginx 1.14.0 on Ubuntu 18.04.3 and everything is up to date.

Ideas?

[bigwhale · Accepted solution]

Just a follow up, after I disabled HTTP2 everything worked and I was able to install my new license. When my license was renewed, I re-enabled the HTTP2 and Jira is working without any problems.

So, the problem is only with the ConfirmNewInstallationWithOldLicense!default.jspa file it's doing something differently than the rest of the system.

My problem is now solved, but someone from Atlassian should really look this up. :)

[Andy Heinzer]

Hi,

Sorry to hear there are problems here.  However from looking at your Jira configuration, I don't see any problems with the way Jira itself is configured.  Instead I found a few things off in the nginx configuration from what I would expect to see.

We have a KB on what we generally expect to see over in Configure Jira server to run behind a NGINX reverse proxy.  From that guide, we expect to see something like this:

server {
    listen www.atlassian.com:80;
    server_name www.atlassian.com;
    location /jira {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
        proxy_pass http://jira-hostname:8080/jira;
        client_max_body_size 10M;
    }
}

Your config listed here does not appear to have proxy_set_header X-Forwarded-Host or proxy_set_header X-Forwarded-Server parameters at all.  But also the value you have in proxy_pass seems different than what I was expecting. 

Your config has

proxy_pass http://jira;

but I'd expect it to be:

proxy_pass http://jira-hostname:8080;

I also noted that our config does not appear to have an upstream section like yours does.  Also we seem to be handling http (port 80) redirection a bit differently than you are here.  Perhaps these are factors in regards to the requests being processed in an unexpected way by Jira.  Step 3 in the guide has more info on the redirection we would expect here for http -> https:

server {
        listen  80 default_server;
        listen  [::]:80 default_server ipv6only=on;
        server_name atlassian.com www.atlassian.com;
        return  301 https://$server_name$request_uri;
}

I'd be interested to see if perhaps you can try to adjust your nginx config to be closer to our KB to see if this helps. Try making these adjustments, restart both nginx and Jira and see if this helps at all. Please let me know the results.

Andy

[bigwhale]

Hi Andy,

thanks for your reply. Unfortunately I already tried your suggestion and it didn't work. My Nginx setup isn't special in any way. However, I have now upgraded to 8.4.2 and tried again with the default configuration and I am still getting the same error. 

Right now in Brave browser, I get this error in the console:

'GET https://jira.lubica.net/static-assets/jquery-min.js net::ERR_HTTP2_PROTOCOL_ERROR 200'

I have a feeling that this has something to do with HTTP2 that is enabled on the server. The fun part is that this very same configuration was working with 7.x version of Jira.