Can we run elastic search "individually" on each Bitbucket cluster?

[SATOSHI ISOMATSU]

Hi 

I've been using Bitbucket Data Center version and I installed elastic search on the local instance. Then, 2nd cluster connects to the elastic search which is installed at 1st cluster. However, this is a risk to lost the functionality of elastic search when the 1st cluster (instance) is down.

My question is:

Is it OK to install and run elastic search on each cluster individually (no replication) ?

The reason why I do not want to use replication setting is to simplify those scale out.

As my guessing, it will consume CPU and memory more for indexing but the result would be same.

Please let me know if there is any issues.

Thanks.

[Caterina Curti · Accepted solution]

Hi Satoshi,

You should only use one single Elasticsearch for Bitbucket Server, all Bitbucket Server nodes will then connect to it via the settings specified in the shared bitbucket.properties file. The instructions are available on the Install and configure a remote Elasticsearch instance page.

Elasticsearch also provides the ability to run in a clustered installation. Bitbucket Server can connect to a clustered Elasticsearch. From our documentation:

Bitbucket Data Center can have only one remote connection to Elasticsearch for your cluster. This may be a standalone Elasticsearch installation or a clustered installation behind a load balancer. 

@Craig Castle-Mead's suggestion to use an additional server for Elasticsearch is also a valid one to avoid issues if a node of the cluster is unavailable.

Thanks, Caterina - Atlassian 

[Craig Castle-Mead]

Hi Satoshi,

In my opinion, even if this is "possible", it should not be done. While Elasticsearch "only" powers the search and if each node will build it's own ES cache if you start a copy of elasticsearch on each node, I feel it's very likely they'll then get out of sync as an action on Node 1 will update Node 1's index, but then Node 2's search is going to be out of sync. It's definitely against the recommended setup on https://confluence.atlassian.com/bitbucketserver/clustering-with-bitbucket-data-center-776640164.html which indicates a shared independent ES environment.

Your goal of getting ES off your one single instance is definitely correct, however the suggestion would be to configure a new server running just elasticsearch and point all the nodes there. If you're running this in AWS then their managed ElasticSearch service is a great way to solve this need.

CCM