http status 404 after hard reboot of computer system (running Ubuntu)

A B May 11, 2012

I hard rebooted my Ubuntu computer that was running my JIRA installation (which has been working fine for 6 months now). And I can't get JIRA to run (even if I do /etc/init.d/jira stop ; /etc/init.d/jira start). The /opt/atlassian/jira/logs/catalina.out log says:

****************

JIRA starting...

****************

2012-05-11 10:02:22,735 main INFO [atlassian.jira.startup.JiraStartupLogger]

___ Environment _____________________________

JIRA Build : 5.0#713-sha1:aec58e27a0d402341b1f1be2fc8c0204a0b070f6

Build Date : Wed Feb 15 00:00:00 EST 2012

JIRA Installation Type : Standalone

Application Server : Apache Tomcat/6.0.32 - Servlet API 2.5

Java Version : 1.6.0_26 - Sun Microsystems Inc.

Current Working Directory : /opt/atlassian/jira/bin

Maximum Allowable Memory : 682MB

Total Memory : 245MB

Free Memory : 182MB

Used Memory : 63MB

Memory Pool: Code Cache : Code Cache: init = 2555904(2496K) used = 1102400(1076K) committed = 2555904(2496K) max = 50331648(49152K)

Memory Pool: PS Eden Space : PS Eden Space: init = 67174400(65600K) used = 66079888(64531K) committed = 67174400(65600K) max = 246153216(240384K)

Memory Pool: PS Survivor Space : PS Survivor Space: init = 11141120(10880K) used = 0(0K) committed = 11141120(10880K) max = 11141120(10880K)

Memory Pool: PS Old Gen : PS Old Gen: init = 178978816(174784K) used = 0(0K) committed = 178978816(174784K) max = 536870912(524288K)

Memory Pool: PS Perm Gen : PS Perm Gen: init = 21757952(21248K) used = 16248392(15867K) committed = 21757952(21248K) max = 268435456(262144K)

JVM Input Arguments : -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -XX:MaxPermSize=256m -Xms256m -Xmx768m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -XX:+PrintGCDateStamps -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/atlassian/jira/endorsed -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp

___ Java System Properties _________________
atlassian.standalone : JIRA
catalina.base : /opt/atlassian/jira
catalina.home : /opt/atlassian/jira
catalina.useNaming : true
common.loader : ${catalina.base}/lib,
${catalina.base}/lib/*.jar,
${catalina.home}/lib,
${catalina.home}/lib/*.jar
file.encoding : UTF-8
file.encoding.pkg : sun.io
java.awt.graphicsenv : sun.awt.X11GraphicsEnvironment
java.awt.headless : true
java.awt.printerjob : sun.print.PSPrinterJob
java.class.version : 50.0
java.home : /opt/atlassian/jira/jre
java.io.tmpdir : /opt/atlassian/jira/temp
java.naming.factory.initial : org.apache.naming.java.javaURLContextFactory
java.naming.factory.url.pkgs : org.apache.naming
java.runtime.name : Java(TM) SE Runtime Environment
java.runtime.version : 1.6.0_26-b03
java.specification.name : Java Platform API Specification
java.specification.vendor : Sun Microsystems Inc.
java.specification.version : 1.6
java.util.logging.config.file : /opt/atlassian/jira/conf/logging.properties
java.util.logging.manager : org.apache.juli.ClassLoaderLogManager
java.vendor : Sun Microsystems Inc.
java.vendor.url : http://java.sun.com/
java.version : 1.6.0_26
java.vm.info : mixed mode
java.vm.name : Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.name : Java Virtual Machine Specification
java.vm.specification.vendor : Sun Microsystems Inc.
java.vm.specification.version : 1.0
java.vm.vendor : Sun Microsystems Inc.
java.vm.version : 20.1-b02
mail.mime.decodeparameters : true
org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER : true
os.arch : amd64
os.name : Linux
os.version : 2.6.32-27-generic
package.access : sun.,
org.apache.catalina.,
org.apache.coyote.,
org.apache.tomcat.,
org.apache.jasper.
server.loader :
shared.loader :
sun.arch.data.model : 64
sun.boot.library.path : /opt/atlassian/jira/jre/lib/amd64
sun.cpu.endian : little
sun.cpu.isalist :
sun.io.unicode.encoding : UnicodeLittle
sun.java.command : org.apache.catalina.startup.Bootstrap start
sun.java.launcher : SUN_STANDARD
sun.jnu.encoding : UTF-8
sun.management.compiler : HotSpot 64-Bit Tiered Compilers
sun.os.patch.level : unknown
tomcat.util.buf.StringCache.byte.enabled : true
user.country : AU
user.dir : /opt/atlassian/jira/bin
user.home : /home/jira
user.language : en
user.name : jira
user.timezone : Australia/Sydney
2012-05-11 10:02:22,736 main INFO [atlassian.jira.startup.JiraStartupLogger] Running JIRA startup checks.
2012-05-11 10:02:22,787 main INFO [atlassian.jira.startup.JiraHomeStartupCheck] The jira.home directory '/var/atlassian/application-data/jira' is validated and locked for exclusive use by this instance.
2012-05-11 10:02:22,788 main INFO [atlassian.jira.startup.JiraStartupLogger] JIRA pre-database startup checks completed successfully.
2012-05-11 10:02:22,963 main INFO [jira.config.database.SystemTenantDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/application-data/jira/dbconfig.xml
2012-05-11 10:02:23,054 main INFO [jira.config.database.DatabaseConfigurationManagerImpl] Now running Database Checklist Launcher
2012-05-11 10:02:23,229 main ERROR [NoModule] Error getting datasource via DBCP: JdbcDatasourceInfo{uri='jdbc:hsqldb:/var/atlassian/application-data/jira/database/jiradb', driverClassName='org.hsqldb.jdbcDriver', username='sa', password='********', isolationLevel='null', connectionProperties=null, connectionPoolInfo=ConnectionPoolInfo{maxSize=20, minSize=2, maxWait=60000, sleepTime=300000, lifeTime=600000, deadLockMaxWait=600000, deadLockRetryWait=10000, validationQuery='null', minEvictableTimeMillis=4000, timeBetweenEvictionRunsMillis=5000}}
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (User not found: SA)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.setLogWriter(BasicDataSource.java:1134)
at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.getConnection(DBCPConnectionFactory.java:123)
at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:69)
at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:146)
at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:101)
at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59)
at com.atlassian.jira.ofbiz.DefaultOfBizConnectionFactory.getConnection(DefaultOfBizConnectionFactory.java:28)
at com.atlassian.jira.upgrade.util.UpgradeUtils.tableExists(UpgradeUtils.java:100)
at com.atlassian.jira.appconsistency.db.MinimumUpgradableVersionCheck.databaseSetup(MinimumUpgradableVersionCheck.java:49)
at com.atlassian.jira.appconsistency.db.MinimumUpgradableVersionCheck.isOk(MinimumUpgradableVersionCheck.java:30)
at com.atlassian.jira.startup.JiraDatabaseConfigChecklist.doStartupChecks(JiraDatabaseConfigChecklist.java:65)
at com.atlassian.jira.startup.JiraDatabaseConfigChecklist.startupOK(JiraDatabaseConfigChecklist.java:53)
at com.atlassian.jira.startup.DatabaseChecklistLauncher.start(DatabaseChecklistLauncher.java:35)
at com.atlassian.jira.startup.DefaultJiraLauncher$2.run(DefaultJiraLauncher.java:90)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:250)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseConfigured(DatabaseConfigurationManagerImpl.java:155)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:85)
at com.atlassian.jira.startup.DefaultJiraLauncher.access$100(DefaultJiraLauncher.java:24)
at com.atlassian.jira.startup.DefaultJiraLauncher$1.run(DefaultJiraLauncher.java:61)
at com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:33)
at com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:56)
at com.atlassian.jira.startup.LauncherContextListener$1.create(LauncherContextListener.java:67)
at com.atlassian.jira.startup.LauncherContextListener$1.create(LauncherContextListener.java:62)
at com.atlassian.multitenant.impl.MultiTenantComponentMapImpl.get(MultiTenantComponentMapImpl.java:121)
at com.atlassian.multitenant.impl.MultiTenantComponentMapImpl.onTenantStart(MultiTenantComponentMapImpl.java:165)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager$1.consume(DefaultMultiTenantManager.java:134)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager$1.consume(DefaultMultiTenantManager.java:131)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.runForEachListener(DefaultMultiTenantManager.java:256)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.startTenant(DefaultMultiTenantManager.java:130)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.startAll(DefaultMultiTenantManager.java:203)
at com.atlassian.jira.startup.LauncherContextListener.contextInitialized(LauncherContextListener.java:95)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.sql.SQLException: User not found: SA
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
at org.hsqldb.jdbcDriver.connect(Unknown Source)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
... 47 more
2012-05-11 10:02:23,233 main ERROR [atlassian.jira.startup.LauncherContextListener] Unable to start JIRA.
com.atlassian.jira.exception.DataAccessException: java.sql.SQLException: User not found: SA
at com.atlassian.jira.upgrade.util.UpgradeUtils.tableExists(UpgradeUtils.java:127)
at com.atlassian.jira.appconsistency.db.MinimumUpgradableVersionCheck.databaseSetup(MinimumUpgradableVersionCheck.java:49)
at com.atlassian.jira.appconsistency.db.MinimumUpgradableVersionCheck.isOk(MinimumUpgradableVersionCheck.java:30)
at com.atlassian.jira.startup.JiraDatabaseConfigChecklist.doStartupChecks(JiraDatabaseConfigChecklist.java:65)
at com.atlassian.jira.startup.JiraDatabaseConfigChecklist.startupOK(JiraDatabaseConfigChecklist.java:53)
at com.atlassian.jira.startup.DatabaseChecklistLauncher.start(DatabaseChecklistLauncher.java:35)
at com.atlassian.jira.startup.DefaultJiraLauncher$2.run(DefaultJiraLauncher.java:90)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:250)
at com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseConfigured(DatabaseConfigurationManagerImpl.java:155)
at com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:85)
at com.atlassian.jira.startup.DefaultJiraLauncher.access$100(DefaultJiraLauncher.java:24)
at com.atlassian.jira.startup.DefaultJiraLauncher$1.run(DefaultJiraLauncher.java:61)
at com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:33)
at com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:56)
at com.atlassian.jira.startup.LauncherContextListener$1.create(LauncherContextListener.java:67)
at com.atlassian.jira.startup.LauncherContextListener$1.create(LauncherContextListener.java:62)
at com.atlassian.multitenant.impl.MultiTenantComponentMapImpl.get(MultiTenantComponentMapImpl.java:121)
at com.atlassian.multitenant.impl.MultiTenantComponentMapImpl.onTenantStart(MultiTenantComponentMapImpl.java:165)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager$1.consume(DefaultMultiTenantManager.java:134)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager$1.consume(DefaultMultiTenantManager.java:131)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.runForEachListener(DefaultMultiTenantManager.java:256)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.startTenant(DefaultMultiTenantManager.java:130)
at com.atlassian.multitenant.impl.DefaultMultiTenantManager.startAll(DefaultMultiTenantManager.java:203)
at com.atlassian.jira.startup.LauncherContextListener.contextInitialized(LauncherContextListener.java:95)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.sql.SQLException: User not found: SA
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
at org.hsqldb.jdbcDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:87)
at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:146)
at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:101)
at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59)
at com.atlassian.jira.ofbiz.DefaultOfBizConnectionFactory.getConnection(DefaultOfBizConnectionFactory.java:28)
at com.atlassian.jira.upgrade.util.UpgradeUtils.tableExists(UpgradeUtils.java:100)
... 38 more
11/05/2012 10:02:23 AM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
11/05/2012 10:02:23 AM org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
11/05/2012 10:02:23 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named [Thread-2] but has failed to stop it. This is very likely to create a memory leak.
11/05/2012 10:02:23 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named [HSQLDB Timer @1702679a] but has failed to stop it. This is very likely to create a memory leak.
11/05/2012 10:02:23 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@41e3a0ec]) and a value of type [org.dom4j.DocumentFactory] (value [org.dom4j.DocumentFactory@475a07bf]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
11/05/2012 10:02:23 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
11/05/2012 10:02:23 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 2546 ms
------------------------
So the first error I see in that log appears to be the database:
[NoModule] Error getting datasource via DBCP: JdbcDatasourceInfo{uri='jdbc:hsqldb:/var/atlassian/application-data/jira/database/jiradb',
But how do I fix that?

4 answers

4 votes
Dieter
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 11, 2012
The problem is hsqldb. When shut down in the wrong moment, you will loose data and corrupt the database. Atlassian warns in all possible places not to use hsqldb productively. Please also check this: http://confluence.atlassian.com/pages/viewpage.action?pageId=119185. The only chance i see is to restore a backup AND to switch to a database like MySQL or PostgreSQL which is free like HSQLDB
0 votes
A B May 11, 2012

Answer to my points:

* you just need to remove /var/atlassian/application-data/jira after *uninstall*ing jira...

* just copy your last backup.zip to /var/atlassian/application-data/jira/import and login to jira and go to the restore JIRA page..

0 votes
A B May 11, 2012

I was thinking that was the issue. However I have the following questions:

* The atlassian-jira-5.0.4-x64.bin doesn't have an option to select a MySQL database, and it seems I need a working JIRA installation to get JIRA setup with MySQL as outlined in http://confluence.atlassian.com/display/JIRA/Setting+Up+a+MySQL+Database+on+Linux+for+JIRA - however I each time I do a fresh re-install - after stopping and removing the old broken HSQLDB installation - I get the same HSQLDB error - so how can I re-install a fresh copy of atlassian-jira-5.0.4-x64.bin ?

* I assume once I have JIRA 5.0.4 up and running with a MySQL database, I can import the backups from my old installation (which used HSQLDB)?

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 11, 2012

I've seen exactly the same - you have to shut down Jira cleanly well before you reboot or HSQL gets corrupted.

You might be able to rescue the database with a little inventive hacking, but there's a good chance that even if you can get back in, you'll have lost data.

As Dieter says, HSQL is not suitable for production instances, so you should treat this as a learning experience, swap this dev/test system to PostgreSQL or MySQL and rebuild it from your production data.

Suggest an answer

Log in or Sign up to answer