Intermittent com.zaxxer.hikari.pool.PoolBase bitbucket errors occuring and connection times out.

sanan sanan

Hi, 

We are running Bitbucket server version 7.12 connected to Oracle 12c. 

For last few weeks we have been receiving reports from the users that their builds and repo cloning, sometimes even pull and push fails with time out errors. 

when I see the logs I find below error messages which matches the time-stamp provided by the users when they see the issue.

 

************

2021-11-05 01:07:42,208 WARN [https-jsse-nio-8443-exec-3254] @RLOAX9x67x25513659x10 192.168.252.113,172.22.105.14 "GET /scm/rt247/participant-registry.git/info/refs HTTP/1.1" com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@75435f98 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:07:42,257 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@24ace393 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:07:42,820 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@384295c0 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:07:42,964 WARN [https-jsse-nio-8443-exec-3238] @RLOAX9x67x25513660x11 192.168.252.113,172.22.105.15 "HEAD / HTTP/1.1" com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@43c98f37 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:07:43,748 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@151efe74 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:07:45,030 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@7d00837f (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:08:55,469 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@6cb4cf19 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:09:05,396 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@64c60127 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.
2021-11-05 01:09:05,410 WARN [ssh-auth] com.zaxxer.hikari.pool.PoolBase bitbucket - Failed to validate connection oracle.jdbc.driver.T4CConnection@4d0393d1 (IO Error: Invalid Operation, NOT Connected). Possibly consider using a shorter maxLifetime value.

***********

I contacted my db team and they say they do not find any problem on the db servers. 

 

What can be the issue here? do I need to increase time-out parameter somewhere in Bitbucket for this? 

Any help will be highly appreciated.

 

Regards

sanan

Nic Brough -Adaptavist-

This usually means that the default value is set differently (higher) than the database's wait_timeout.

I've fixed it in the past by changing the database to match (from memory, 30 minutes is the off-the-shelf value, but don't quote me!), but you could equally change Bitbucket's value to match the database setting.  It's a setting in one of the property or xml files that says maxLifeTime if memory serves (I'm pretty sure about the word, but not the capitals in it)