Bitbucket not startup after upgrade from 4.5.2 to 5.10.0

[juan.caldino]

*** bitbucket.properties file***************

#>*******************************************************
#> Migrated to database at jdbc:mysql://10.13.8.141:3306/atlstash?characterEncoding=utf8&useUnicode=true&sessionVariables=storage_engine%3DInnoDB
#> Updated on 2015-04-08T16:07:52.263-06:00
#>*******************************************************
jdbc.driver=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://10.13.8.141:3306/atlstash?characterEncoding=utf8&useUnicode=true&sessionVariables=storage_engine%3DInnoDB
jdbc.user=xxxxx
jdbc.password=xxxxxx

server.compression.enabled=true
server.compression.mime-types=text/css,text/html,text/javascript,text/json,text/plain,text/xml,text/x-javascript,\application/javascript,application/json,application/x-javascript,application/vnd.git-lfs+json
server.connection-timeout=20000
server.context-path=
server.displayName=Atlassian Bitbucket
server.port=7990
server.address=10.13.8.141
server.session.cookie.http-only=true
server.secure=true
server.ssl.enabled=true
#server.ssl.key-alias=tomcat
server.ssl.protocol=TLSv1.2
server.redirect-port=443

[juan.caldino]

2018-05-02 13:31:20,268 INFO [main] o.s.boot.SpringApplication
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.12.RELEASE)

2018-05-02 13:31:20,508 INFO [main] c.a.b.i.b.BitbucketServerApplication Starting BitbucketServerApplication on unity with PID 23283 (/opt/atlassian/bitbucket/5.10.0/app/WEB-INF/classes started by atlbitbucket in /opt/atlassian/bitbucket/5.10.0/bin)
2018-05-02 13:31:20,509 INFO [main] c.a.b.i.b.BitbucketServerApplication No active profile set, falling back to default profiles: default
2018-05-02 13:31:22,333 INFO [main] c.a.b.i.boot.log.BuildInfoLogger Starting Bitbucket 5.10.0 (76ff11a built on Sun Apr 22 20:39:21 MDT 2018)
2018-05-02 13:31:22,333 INFO [main] c.a.b.i.boot.log.BuildInfoLogger JVM: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_05-b13
2018-05-02 13:31:25,304 ERROR [main] o.a.catalina.core.StandardService Failed to start connector [Connector[HTTP/1.1-7990]]
org.apache.catalina.LifecycleException: Failed to start component [Connector[HTTP/1.1-7990]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167)
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.addPreviouslyRemovedConnectors(TomcatEmbeddedServletContainer.java:265)
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.start(TomcatEmbeddedServletContainer.java:208)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.startEmbeddedServletContainer(EmbeddedWebApplicationContext.java:297)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:145)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at com.atlassian.bitbucket.internal.boot.BitbucketServerApplication.start(BitbucketServerApplication.java:247)
at com.atlassian.bitbucket.internal.boot.BitbucketServerApplication.main(BitbucketServerApplication.java:83)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at com.atlassian.bitbucket.internal.launcher.BitbucketServerLauncher.start(BitbucketServerLauncher.java:151)
at com.atlassian.bitbucket.internal.launcher.BitbucketServerLauncher.main(BitbucketServerLauncher.java:99)
... 5 frames trimmed
Caused by: org.apache.catalina.LifecycleException: Protocol handler start failed
at org.apache.catalina.connector.Connector.startInternal(Connector.java:1020)
... 16 common frames omitted
Caused by: java.lang.IllegalArgumentException: Keystore was tampered with, or password was incorrect
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:116)
... 16 common frames omitted
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
at java.security.KeyStore.load(KeyStore.java:1433)
... 16 common frames omitted
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770)
... 18 common frames omitted

[juan.caldino]

2018-05-02 13:31:28,518 WARN [spring-startup] c.a.stash.internal.home.HomeLock Failed to write process information into the lock file
java.nio.channels.ClosedByInterruptException: null
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
at com.atlassian.stash.internal.home.HomeLock.acquireLock(HomeLock.java:123)
at com.atlassian.stash.internal.home.HomeLock.lock(HomeLock.java:94)
at com.atlassian.stash.internal.home.HomeLockAcquirer.lock(HomeLockAcquirer.java:58)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:151)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at java.lang.Thread.run(Thread.java:745)
... 28 frames trimmed
2018-05-02 13:31:28,530 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory /var/atlassian/application-data/stash
2018-05-02 13:31:28,942 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:29,895 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:29,909 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:29,970 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:30,016 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:30,030 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:30,040 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:30,043 ERROR [spring-startup] com.zaxxer.hikari.HikariConfig Failed to load driver class com.mysql.jdbc.Driver from HikariConfig class classloader org.springframework.boot.loader.LaunchedURLClassLoader@4517d9a3
2018-05-02 13:31:30,044 WARN [spring-startup] o.s.w.c.s.XmlWebApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'licenseHelper' while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'licenseHelper' defined in URL [jar:file:/opt/atlassian/bitbucket/5.10.0/app/WEB-INF/lib/bitbucket-service-impl-5.10.0.jar!/com/atlassian/stash/internal/license/DefaultLicenseHelper.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseSupplier' defined in URL [jar:file:/opt/atlassian/bitbucket/5.10.0/app/WEB-INF/lib/bitbucket-dao-impl-5.10.0.jar!/com/atlassian/stash/internal/db/DefaultDatabaseSupplier.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactoryPrototype' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'dataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
2018-05-02 13:31:30,045 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Releasing lock on /var/atlassian/application-data/stash
2018-05-02 13:31:30,049 ERROR [spring-startup] c.a.j.s.w.s.JohnsonDispatcherServlet SpringMVC dispatcher [springMvc] could not be started
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'licenseHelper' while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'licenseHelper' defined in URL [jar:file:/opt/atlassian/bitbucket/5.10.0/app/WEB-INF/lib/bitbucket-service-impl-5.10.0.jar!/com/atlassian/stash/internal/license/DefaultLicenseHelper.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseSupplier' defined in URL [jar:file:/opt/atlassian/bitbucket/5.10.0/app/WEB-INF/lib/bitbucket-dao-impl-5.10.0.jar!/com/atlassian/stash/internal/db/DefaultDatabaseSupplier.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bean 'dataSourceFactory$created#4d449b58' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#4d449b58' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:667)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at java.lang.Thread.run(Thread.java:745)
... 19 frames trimmed

[Hua Soon SIM [Akeles]]

The documentation at https://confluence.atlassian.com/bitbucketserver/connecting-bitbucket-server-to-mysql-776640382.html is correct.

It works when the jar file is copied to <bitbucket home directory>\lib.

The advantage of this is the jar file will persist across Bitbucket upgrades.

It does not work if the jar file is copied to <bitbucket installation directory>\lib

Alternatively, it also works if the MySQL driver is copied to <bitbucket installation directory>\app\WEB-INF\lib which the drivers for other databases are located.

[Matthew Ford]

Thanks!  This helped!