Could not join cluster in 300000 ms Error migrating Database

Question

Trying to perform a database migration from internal postgres to external 9.5 Postgres database we always get the same errors in log /opt/atlassian/bitbucket-data/log/atlassian-bitbucket.log

ERROR [threadpool:thread-4] c.a.s.i.c.StateTransferringExecutor Error while processing asynchronous task
ERROR [cached3] com.hazelcast.instance.Node [3.5.2-atlassian-37] Could not join cluster in 300000 ms. Shutting down now!

We are using  Atlassian Bitbucket v4.11.0 and want upgrade after database migration to 5.1

I created postgres database as doc said:

postgres=# CREATE ROLE bitbucketuser WITH LOGIN PASSWORD 'jellyfish' VALID UNTIL 'infinity';CREATE ROLE

postgres=# CREATE DATABASE bitbucket WITH ENCODING='UTF8' OWNER=bitbucketuser CONNECTION LIMIT=-1;

Thanks

Antonio Cerezo Iglesias · Answer

What finally worked for us was to disable AWS discovery and add the cluster IPs for unicast discovery:

hazelcast.network.aws=false

hazelcast.network.tcpip.members=X.X.X.X

Antonio Cerezo Iglesias · Answer

Thanks Daniel for your quick reply and sharing your solution!

Will check the logs to see if we're hitting the same issue.

Best regards!

Antonio

unknown · Answer

Hi Antonio.

We have cloned the server to test the process, and the new server was still trying to connect with somthing in the old environment.

Have found this in logs:

2017-06-12 08:46:03,927 WARN [cached1] c.h.c.i.o.MergeClustersOperation [192.168.10.188]:5701 [jira] [3.5.2-atlassian-37] Address[192.168.10.188]:5701 is merging to Address[192.168.10.11]:5701, because: instructed by master Address[192.168.10.188]:5701

So i added 2 new iptables rules to block traffic with old server and database  and try it again. And it Works!

Regards