Unexpected exception during JIRA startup. This JIRA instance will not be able to recover.

Question

After realizing that the hard drive was full I expanded the Drive after stopping the JIRA service.  After expanding the volume and restarting the server I confirmed that the JIRA service is started but now we get :  Unexpected exception during JIRA startup. This JIRA instance will not be able to recover. Please check the logs for details.

This is what catalina.2018-01-15 shows:

15-Jan-2018 11:21:36.104 WARNING [http-nio-8080-exec-28] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI http://insjirasrv1.inspirata.lan:8080/rest/gadget/1.0/login, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
15-Jan-2018 13:00:27.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.6
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Oct 6 2016 20:15:31 UTC
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.6.0
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Windows Server 2012 R2
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 6.3
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: C:\PROGRA~1\ATLASS~1\JIRA\jre
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_102-b14
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: C:\Program Files\Atlassian\JIRA
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: C:\Program Files\Atlassian\JIRA
15-Jan-2018 13:00:27.451 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Datlassian.standalone=JIRA
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dmail.mime.decodeparameters=true
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=C:\Program Files\Atlassian\JIRA
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=C:\Program Files\Atlassian\JIRA
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=C:\Program Files\Atlassian\JIRA	emp
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=C:\Program Files\Atlassian\JIRA\conf\logging.properties
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: exit
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms512m
15-Jan-2018 13:00:27.467 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx1538m

Luis A Flores · Answer

Thank you.  Turns out the db was installed with an evaluation copy of SQL Server and had expired.  I was able to activate it and all seems ok now.  Thank you very much for your replies!

Jobin Kuruvilla [Adaptavist] · Answer

As you can see from the error log, you are getting connection refused from the database. Please check if your DB settings are correct or if there is another issue preventing the connection to the DB server.

Luis A Flores · Answer

2018-01-16 07:21:48,375 http-nio-8080-exec-21 ERROR [NoModule] Could not determine database type. java.sql.SQLException: Network error IOException: Connection refused: connect at net.sourceforge.jtds.jdbc.JtdsConnection.(JtdsConnection.java:436) at net.sourceforge.jtds.jdbc.Driver.connect(Driver.java:184) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:87) at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:164) at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:136) at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59) at org.ofbiz.core.entity.config.DatasourceInfo.getDatabaseTypeFromJDBCConnection(DatasourceInfo.java:402) at org.ofbiz.core.entity.GenericDAO.selectListIteratorByCondition(GenericDAO.java:831) at org.ofbiz.core.entity.GenericDAO.selectByAnd(GenericDAO.java:730) at org.ofbiz.core.entity.GenericHelperDAO.findByAnd(GenericHelperDAO.java:150) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:903) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:881) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:850) at com.opensymphony.module.propertyset.ofbiz.OFBizPropertySet.findPropertyEntry(OFBizPropertySet.java:298) at com.opensymphony.module.propertyset.ofbiz.OFBizPropertySet.exists(OFBizPropertySet.java:170) at com.atlassian.jira.propertyset.JiraCachingPropertySet.exists(JiraCachingPropertySet.java:566) at com.atlassian.jira.propertyset.BootstrapOfBizPropertyEntryStore.getType(BootstrapOfBizPropertyEntryStore.java:67) at com.atlassian.jira.propertyset.CachingOfBizPropertySet.exists(CachingOfBizPropertySet.java:139) at com.atlassian.jira.config.properties.ApplicationPropertiesStore.getOption(ApplicationPropertiesStore.java:157) at com.atlassian.jira.config.properties.ApplicationPropertiesImpl.getOption(ApplicationPropertiesImpl.java:86) ... 1 filtered at java.util.Optional.map(Optional.java:215) ... 22 filtered at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.connect0(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at net.sourceforge.jtds.jdbc.SharedSocket.createSocketForJDBC3(SharedSocket.java:288) at net.sourceforge.jtds.jdbc.SharedSocket.(SharedSocket.java:251) at net.sourceforge.jtds.jdbc.JtdsConnection.(JtdsConnection.java:331) ... 49 more 2018-01-16 07:21:48,377 http-nio-8080-exec-19 WARN [o.a.commons.dbcp2.BasicDataSource] Failed to complete JMX registration javax.management.InstanceAlreadyExistsException: com.atlassian.jira:name=BasicDataSource at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.commons.dbcp2.BasicDataSource.jmxRegister(BasicDataSource.java:2346) at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2029) at org.apache.commons.dbcp2.BasicDataSource.setLogWriter(BasicDataSource.java:1625) at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.getConnection(DBCPConnectionFactory.java:103) at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:69) at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:164) at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:136) at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59) at org.ofbiz.core.entity.config.DatasourceInfo.getDatabaseTypeFromJDBCConnection(DatasourceInfo.java:402) at org.ofbiz.core.entity.GenericDAO.selectListIteratorByCondition(GenericDAO.java:831) at org.ofbiz.core.entity.GenericDAO.selectByAnd(GenericDAO.java:730) at org.ofbiz.core.entity.GenericHelperDAO.findByAnd(GenericHelperDAO.java:150) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:903) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:881) at org.ofbiz.core.entity.GenericDelegator.findByAnd(GenericDelegator.java:850) at com.opensymphony.module.propertyset.ofbiz.OFBizPropertySet.findPropertyEntry(OFBizPropertySet.java:298) at com.opensymphony.module.propertyset.ofbiz.OFBizPropertySet.exists(OFBizPropertySet.java:170) at com.atlassian.jira.propertyset.JiraCachingPropertySet.exists(JiraCachingPropertySet.java:566) at com.atlassian.jira.propertyset.BootstrapOfBizPropertyEntryStore.getType(BootstrapOfBizPropertyEntryStore.java:67) at com.atlassian.jira.propertyset.CachingOfBizPropertySet.exists(CachingOfBizPropertySet.java:139) at com.atlassian.jira.config.properties.ApplicationPropertiesStore.getOption(ApplicationPropertiesStore.java:157) at com.atlassian.jira.config.properties.ApplicationPropertiesImpl.getOption(ApplicationPropertiesImpl.java:86) ... 1 filtered at java.util.Optional.map(Optional.java:215) ... 22 filtered at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) 2018-01-16 07:21:49,399 http-nio-8080-exec-19 ERROR [NoModule] Error getting datasource via DBCP: JdbcDatasourceInfo{uri='jdbc:jtds:sqlserver://localhost:1433/JiraDB', driverClassName='net.sourceforge.jtds.jdbc.Driver', username='jiradbuser', password='********', isolationLevel='null', connectionProperties=null, connectionPoolInfo=ConnectionPoolInfo{maxSize=20, minSize=20, initialSize=null, maxIdle=20, maxWait=30000, sleepTime=300000, lifeTime=600000, deadLockMaxWait=600000, deadLockRetryWait=10000, validationQuery='select 1', minEvictableTimeMillis=60000, timeBetweenEvictionRunsMillis=300000, poolPreparedStatements=null, testOnBorrow=false, testOnReturn=null, testWhileIdle=true, maxOpenPreparedStatements=null, numTestsPerEvictionRun=null, removeAbandonedOnBorrow=true, removeAbandonedOnMaintanance=null, removeAbandonedTimeout=300, validationQueryTimeout=null, defaultCatalog=null}} java.sql.SQLException: Cannot create PoolableConnectionFactory (Network error IOException: Connection refused: connect)