Out of the blue bitbucket stopped working

[Mohanad Tayem]

yesterday we were trying to get a repo from our bitbucket only to find ,that it had crashed and was giving the following message 

we tried restarting service , it always hang on Migrating home directory , before showing the fatal error, i did check connection to db everything works fine, also tried DATABASECHANGELOGLOCK with no luck, could someone help please ?

[Nic Brough -Adaptavist-]

This means your Bitbucket server cannot connect to the database you have configured it to use.  It could be that the database is not set up correctly, but the messages you are having suggest that it's more likely to be that the database simply isn't letting you into it.  The messages say something is coming back from it, but "communications link failure" says that the responses are not a lot more than "I'm configured not to talk to you"

[Mohanad Tayem]

The last packet successfully received from the server was 1 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
... 6 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [stash-context.xml]: Cannot create inner bea
n 'dataSourceFactory$created#41bafa23' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dat
aSourceFactory$created#41bafa23' defined in class path resource [stash-context.xml]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInsta
ntiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationExceptio
n: Failed to initialize pool: Communications link failure

The last packet successfully received from the server was 1 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:389)
... 5 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSourceFactory$created#41bafa23' defined in class path resource [stash-context.xml]:
Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'cre
ate' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Communications link failure

The last packet successfully received from the server was 1 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:658)
... 5 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'create' threw exception; nested exception is com.zaxxer.hik
ari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Communications link failure

The last packet successfully received from the server was 1 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
... 5 common frames omitted
Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Communications link failure

The last packet successfully received from the server was 1 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:596)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:582)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at com.atlassian.stash.internal.hikari.ExtendedHikariDataSource.<init>(ExtendedHikariDataSource.java:49)
at com.atlassian.stash.internal.hikari.HikariDataSourceFactory.create(HikariDataSourceFactory.java:70)
... 5 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

found this also in logs