how to prevent GBs of log files

I installed the Bitbucket server maybe a month ago, using it privately only for my test / sample projects.

Instead of using the default path for the repository I used a folder on my OneDrive. Recently I have noticed that my OneDrive is constantly syncing, and today when I finally looked into it I was surprised to see 2 file in the log folder, each at 9.5GB , and worst of all, one of them was changing even when there was not activity.

I have now reinstalled BitBucket, using the default path instead of OneDrive. But why does it generate 2 huge files like this (it seems related to the elastic server - now sure what that is)

And question is - how can I disable logging, and what is elastic search, is it something I need ?

Thanks

Update - after having it running overnight, with no activity, the elasticsearch has produced 113GB !!! of logs. crazylog.png

 

Any suggestions of how to disable this (orther than disable the service) ?

 

5 answers

This widget could not be displayed.

Hi Jesper,

 

You might want to rotate the logs and create a cron job to remove/archive old logs every week or months based on your preferences. I've found similar threat below discussing a similar issue whereby logs are piling up  :

https://answers.atlassian.com/questions/323782/jira-log-rotation

https://answers.atlassian.com/questions/200444

Hope that will be able to shed some light.

 

Regards,
Jonathan 

This widget could not be displayed.

Could you please post some details about what is in your logs? The log errors could be because you haven't set up an Elastic search instance. In Bitbucket Server 4.6, the code search feature was released. This required a separate process (elasticsearch) to be started along with Bitbucket Server. Depending on how you installed and started Bitbucket Server, you may already have elasticsearch installed but just haven't started the service yet. If you don't have elasticsearch installed at all, you can take a look at the documentation here to install it. 

Once elasticsearch is configured properly, I would expect the logging to go down. However, without seeing any snippets of your logs there is no way of knowing that for sure.

This widget could not be displayed.

I restarted the service. After 1/2 hour it hard produced 2 logs, each of 3.8GB.

The huge size of the log is due to an exception which is thrown (and traced) 2.000.000 times in 1/2 hour. See attach images and text.

Remember, similar issue happened when first installed using the onedrive for data path . I then reinstalled it using default path.

--- seach of log file —

atlbitbucket-elasticsearch-x64-stdout.2016-06-19.log(31287550): [bitbucket-search-v1][[bitbucket-search-v1][1]] IndexShardRecoveryException[failed to recovery from gateway]
Found 2032426 occurrence(s) in 1 file(s)

2502016-06-19_200116_one_hour_run.png2016-06-19_200116_one_hour_run_1.png

This widget could not be displayed.

These 2 exceptions are the ones that fill the log

 

[bitbucket-search-v1][[bitbucket-search-v1][3]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:250)


 at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)


 at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)


 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)


 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)


 at java.lang.Thread.run(Unknown Source)


Caused by: [bitbucket-search-v1][[bitbucket-search-v1][3]] EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:155)


 at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)


 at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1515)


 at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1499)


 at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:972)


 at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:944)
This widget could not be displayed.

This sounds like there is a problem with your search home directory. I would recommend opening a support ticket at http://support.atlassian.com/ so that Atlassian Support is able to do a more thorough investigation.

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Aug 21, 2018 in Bitbucket

Branch Management with Bitbucket

As a project manager, I have discovered that different developers want to bring their previous branching method with them when they join the team. Some developers are used to performing individual wo...

1,201 views 8 10
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you