change db connection to ?ssl=true causes errors

Question

We're trying to enforce the db connection to use ssl with the ?ssl=true setting and jira throws errors when starting.  We've taken the same environment (jira app server and db) to a test environment and everything works fine.

The production server still fails to start correctly

Below is from the catalina.out log file which appears to connect to the db correctly but as you can see there's errors

2017-12-05 01:50:27,427 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/application-data/jira/dbconfig.xml
2017-12-05 01:50:28,358 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] Running JIRA startup checks.
2017-12-05 01:50:28,358 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] JIRA pre-database startup checks completed successfully.

2017-12-05 01:50:28,536 http-nio-8080-exec-2 ERROR - [o.a.c.c.C.[.[localhost].[/].[action]] Servlet.service() for servlet action threw exception
java.lang.NoClassDefFoundError: Could not initialize class webwork.dispatcher.GenericDispatcher
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.prepareDispatcher(JiraWebworkActionDispatcher.java:196)
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(JiraWebworkActionDispatcher.java:133)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
... 53 filtered
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
2017-12-05 01:50:28,536 http-nio-8080-exec-3 ERROR - [o.a.c.c.C.[.[localhost].[/].[action]] Servlet.service() for servlet action threw exception
java.lang.NoClassDefFoundError: Could not initialize class webwork.dispatcher.GenericDispatcher
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.prepareDispatcher(JiraWebworkActionDispatcher.java:196)
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(JiraWebworkActionDispatcher.java:133)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
... 53 filtered
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
2017-12-05 01:50:28,540 http-nio-8080-exec-1 ERROR - [o.a.c.c.C.[.[localhost].[/].[action]] Servlet.service() for servlet action threw exception
java.lang.NullPointerException
at com.atlassian.jira.config.webwork.DelegatingConfiguration.getImpl(DelegatingConfiguration.java:26)
at webwork.config.Configuration.get(Configuration.java:44)
at webwork.config.Configuration.getString(Configuration.java:54)
at webwork.dispatcher.GenericDispatcher.<clinit>(GenericDispatcher.java:43)
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.prepareDispatcher(JiraWebworkActionDispatcher.java:196)
... 1 filtered
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
... 55 filtered
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

2017-12-05 01:50:28,715 JIRA-Bootstrap INFO [o.o.c.entity.jdbc.DatabaseUtil] Database Product Name is PostgreSQL
2017-12-05 01:50:28,716 JIRA-Bootstrap INFO [o.o.c.entity.jdbc.DatabaseUtil] Database Product Version is 9.6.2
2017-12-05 01:50:28,716 JIRA-Bootstrap INFO [o.o.c.entity.jdbc.DatabaseUtil] Database Driver Name is PostgreSQL Native Driver
2017-12-05 01:50:28,717 JIRA-Bootstrap INFO [o.o.c.entity.jdbc.DatabaseUtil] Database Driver Version is PostgreSQL 9.4.1212

john morrissey · Answer

Branden,

Yes we were seeing an internal server error page which a restart of jira doesn't fix it.  We have to revert the db connection to use non ssl to get jira to come up.  The strange thing is the db is connected just fine.

Anyway after a lot of testing duplicating the exact environment we couldn't get the issue to occur in the test environment.

We found our AWS RDS postgresql instance was listing it needed a reboot from a db parameter change.  Once we rebooted the instance jira was able to run with the db ssl parameter set.

So it seems our AWS RDS was in a "funky state" which the reboot took care of it.

somethingblue · Answer

Hi John,

* What version of JIRA do you use?
* Does a restart resolve the issue?
* Do you see a page that says JIRA Internal Error?

The stack trace you provided matches a bug that you can find at JRASERVER-62486.  This is reportedly fixed in 7.6.1 per the notes in the comments.

The workaround is as follows:

Block web access to the instance during start up to prevent the bad request from hitting JIRA during startup such as the following examples:

* Change the port in server.xml during startup or upgrade to avoid users unintentionally accessing URLs which may cause such problems

* Leverage a firewall on the server hosting JIRA to block the port during startup

* In cases where a reverse proxy or balancer is in front of JIRA this can be shut down during startup

Take a look at JRASERVER-62486 and if you are able to get around the issue by using any of the steps provided above please let me know and vote on JRASERVER-62486.

Cheers,

Branden