BitBucket 4.8.1 - Cant start atlbitbucket_search?

[Mark Millette]

Need a little help, I cloned our DB server and stash server to test an in-place upgrade of stash 3.6 to Bitbucket 4.8.1. I am able to login to the upgraded version, but have 2 issues I cant seem to resolve.

1) Search feature not working or starting. From the info below you can see that it claims the service started, but if i do a service status, it shows not running

2) The log4j configuration 

root@test-stash:/var/stash-home/shared# service atlbitbucket_search start

Starting Elasticsearch bundled with Atlassian Bitbucket

BITBUCKET_HOME set to /var/stash-home

Using BITBUCKET_HOME:      /var/stash-home

Starting Elasticsearch bundled with Atlassian Bitbucket as dedicated user atlbitbucket 

Elasticsearch bundled with Atlassian Bitbucket started successfully

root@test-stash:/var/stash-home/shared# log4j:WARN No appenders could be found for logger (bootstrap).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

I must admit I am not all that familiar with this suite of software, so If someone could point me in the right direct, or things to look for that would be greatly appreciated.

Mainly what config files are handling the Search feature and do we need the log4j in version 4.8.1? If so what am I missing? I have read lots of KBs on the site but nothing referencing the Bitbucket.

Thanks,

Mark

[Jeff Thomas]

Hi Mark,

I think you may have hit https://jira.atlassian.com/browse/BSERV-9056

Can you follow the workaround listed there to copy over the configuration files into BITBUCKET_HOME and then restart Elasticsearch?

[Mark Millette]

Thanks Jeff, "BSERV-9056 did solve my issue!

-Mark

[Mark Millette]

Jeff,

The steps outlined in BSERV-9056 has solved the search issue!

Thank you,

Mark

[Mark Millette]

Jeff,

Sorry for the delay on getting back to this. I have checked the $BITBUCKET_HOME/log/search/ only to find no file has been created.

Here is the output of ps -ef | grep elasticsearch

root@test-stash:/var/stash-home/log/search# ps -ef | grep elasticsearch

atlbitb+  1763     1  0 Aug10 ?        00:26:33 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+  2955     1  0 Aug10 ?        00:19:13 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+  4971     1  0 Aug10 ?        00:19:24 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+  8410     1  0 Aug10 ?        00:18:59 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+  8922     1  0 Aug10 ?        00:19:23 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+  9528     1  0 Aug10 ?        00:19:08 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+ 11184     1  0 Aug10 ?        00:19:03 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

atlbitb+ 14244     1  0 Aug10 ?        00:18:41 /usr/lib/jvm/java-8-openjdk-amd64//bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/opt/atlassian/bitbucket/4.8.1/elasticsearch -cp /opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/elasticsearch-2.3.1.jar:/opt/atlassian/bitbucket/4.8.1/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /var/stash-home/shared/search/elasticsearch.pid -Dpath.conf=/var/stash-home/shared/search -Dpath.logs=/var/stash-home/log/search -Dpath.data=/var/stash-home/shared/search/data

root     28170 28039  0 09:15 pts/1    00:00:00 grep elasticsearch

As to the buckler.yml file, it is not in $BITBUCKET_HOME/shared/search/buckler/buckler.yml

But i did find a copy in...

/opt/atlassian/bitbucket/4.8.1/elasticsearch/config-template/buckler/buckler.yml

Thanks,

Mark

[Jeff Thomas]

Hi Mark,

Can you check the $BITBUCKET_HOME/log/search/bitbucket_search.log to see if there are any errors on startup? After search has started, what is the output of ps -ef | grep elasticsearch? Also, is the $BITBUCKET_HOME/shared/search/buckler/buckler.yml present?