BitBucket database connection fails every 30 minutes Edited

Hi, I am writing to you because I have a problem with my BitBucket production server. Over the weekend it was upgrade from Stash 3.0.1 to BitBucket 5.6.1 in 3 steps:
- upgrade from 3.0.1 to 3.11
- upgrade from 3.11 to 4.14
- upgrade from 4.14 to 5.6.1

and soon after that every 30 minutes I lose database connection and UI stops working. My BitBucket is integrated with JIRA and uses internal HSQL database. I recreated application link with JIRA and it is configured to use from 512MB to 4GB RAM.  

Please let me know if you have any idea what can be wrong. Thank you.

I receive the following exceptions:

2017-12-12 09:22:10,385 ERROR [http-nio-7990-exec-80] me @182YHMSx561x6902x112 g633tk 10.180.102.130 "POST /rest/api/latest/projects/AMMLKYC/repos/ammlkyc/pull-requests/2281/comments HTTP/1.1" o.h.e.jdbc.spi.SqlExceptionHelper bitbucket - Connection is not available, request timed out after 15000ms.
2017-12-12 09:22:10,387 DEBUG [http-nio-7990-exec-80] me @182YHMSx561x6902x112 g633tk 10.180.102.130 "POST /rest/api/latest/projects/AMLKYC/repos/amlkyc/pull-requests/2281/comments HTTP/1.1" c.a.s.i.r.e.ServiceExceptionMapper Mapping ServiceException to REST response 500
com.atlassian.bitbucket.DataStoreException: A database error has occurred.
at com.atlassian.stash.internal.aop.ExceptionRewriteAdvice.afterThrowing(ExceptionRewriteAdvice.java:45)
at com.atlassian.stash.internal.permission.GrantedPermissionVoterProvider.lambda$loadGroupsByUser$2(GrantedPermissionVoterProvider.java:238)
..
at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.lang.Thread.run(Thread.java:745)
... 297 frames trimmed
Caused by: org.hibernate.exception.JDBCConnectionException: could not prepare statement
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:48)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
..
at com.atlassian.stash.internal.crowd.RiotPolice.findGroupsByUser(RiotPolice.java:301)
at com.atlassian.stash.internal.user.DefaultUserService.findGroupsByUser(DefaultUserService.java:200)
... 64 common frames omitted
Caused by: java.sql.SQLTransientConnectionException: bitbucket - Connection is not available, request timed out after 15000ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:603)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:193)
..
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:172)
... 95 common frames omitted
2017-12-12 09:22:12,248 ERROR [http-nio-7990-exec-92] @182YHMSx561x6889x109 10.244.16.24 "GET /rest/remote-link-aggregation/latest/aggregation HTTP/1.1" c.a.p.r.c.e.j.ThrowableExceptionMapper Uncaught exception thrown by REST service: There was a SQL exception thrown by the Active Objects library:
null
null

java.sql.SQLTransientConnectionException: bitbucket - Connection is not available, request timed out after 15000ms.
com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
null
null

java.sql.SQLTransientConnectionException: bitbucket - Connection is not available, request timed out after 15000ms.
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.find(EntityManagedActiveObjects.java:142)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.find(TenantAwareActiveObjects.java:296)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)

 

1 answer

1 accepted

Comments for this post are closed

Community moderators have prevented the ability to post new answers.

Post a new question

0 votes
Accepted answer

Hi Rafal,

 

As discussed in the other thread (https://community.atlassian.com/t5/Bitbucket-questions/BitBucket-fails-every-30-minutes/qaq-p/687791), HSQL database is not supported for production instances.

 

If the problem continues after moving to a supported database, we should start by checking the details of the database pool using the JMX counters, in particular the ones related to HikariCP.

 

Caterina - Atlassian Support 

Community showcase
Published Nov 06, 2018 in Bitbucket

Upgrade Best Practices

Hello! My name is Mark Askew and I am a Premier Support Engineer for products Bitbucket Server/Data Center, Fisheye & Crucible. Today, I want to bring the discussion that Jennifer, Matt, and ...

434 views 6 9
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you