how to prevent GBs of log files

[jesperMac]

I installed the Bitbucket server maybe a month ago, using it privately only for my test / sample projects.

Instead of using the default path for the repository I used a folder on my OneDrive. Recently I have noticed that my OneDrive is constantly syncing, and today when I finally looked into it I was surprised to see 2 file in the log folder, each at 9.5GB , and worst of all, one of them was changing even when there was not activity.

I have now reinstalled BitBucket, using the default path instead of OneDrive. But why does it generate 2 huge files like this (it seems related to the elastic server - now sure what that is)

And question is - how can I disable logging, and what is elastic search, is it something I need ?

Thanks

Update - after having it running overnight, with no activity, the elasticsearch has produced 113GB !!! of logs. crazylog.png

Any suggestions of how to disable this (orther than disable the service) ?

[Kristy]

This sounds like there is a problem with your search home directory. I would recommend opening a support ticket at http://support.atlassian.com/ so that Atlassian Support is able to do a more thorough investigation.

[jesperMac]

These 2 exceptions are the ones that fill the log

[bitbucket-search-v1][[bitbucket-search-v1][3]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:250)


 at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)


 at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)


 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)


 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)


 at java.lang.Thread.run(Unknown Source)


Caused by: [bitbucket-search-v1][[bitbucket-search-v1][3]] EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:155)


 at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)


 at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1515)


 at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1499)


 at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:972)


 at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:944)

[jesperMac]

I restarted the service. After 1/2 hour it hard produced 2 logs, each of 3.8GB.

The huge size of the log is due to an exception which is thrown (and traced) 2.000.000 times in 1/2 hour. See attach images and text.

Remember, similar issue happened when first installed using the onedrive for data path . I then reinstalled it using default path.

--- seach of log file —

atlbitbucket-elasticsearch-x64-stdout.2016-06-19.log(31287550): [bitbucket-search-v1][[bitbucket-search-v1][1]] IndexShardRecoveryException[failed to recovery from gateway]
Found 2032426 occurrence(s) in 1 file(s)

2502016-06-19_200116_one_hour_run.png2016-06-19_200116_one_hour_run_1.png

[Kristy]

Could you please post some details about what is in your logs? The log errors could be because you haven't set up an Elastic search instance. In Bitbucket Server 4.6, the code search feature was released. This required a separate process (elasticsearch) to be started along with Bitbucket Server. Depending on how you installed and started Bitbucket Server, you may already have elasticsearch installed but just haven't started the service yet. If you don't have elasticsearch installed at all, you can take a look at the documentation here to install it. 

Once elasticsearch is configured properly, I would expect the logging to go down. However, without seeing any snippets of your logs there is no way of knowing that for sure.

[Jonathan Soo]

Hi Jesper,

You might want to rotate the logs and create a cron job to remove/archive old logs every week or months based on your preferences. I've found similar threat below discussing a similar issue whereby logs are piling up  :

https://answers.atlassian.com/questions/323782/jira-log-rotation

https://answers.atlassian.com/questions/200444

Hope that will be able to shed some light.

Regards,
Jonathan