how to prevent GBs of log files

I installed the Bitbucket server maybe a month ago, using it privately only for my test / sample projects.

Instead of using the default path for the repository I used a folder on my OneDrive. Recently I have noticed that my OneDrive is constantly syncing, and today when I finally looked into it I was surprised to see 2 file in the log folder, each at 9.5GB , and worst of all, one of them was changing even when there was not activity.

I have now reinstalled BitBucket, using the default path instead of OneDrive. But why does it generate 2 huge files like this (it seems related to the elastic server - now sure what that is)

And question is - how can I disable logging, and what is elastic search, is it something I need ?

Thanks

Update - after having it running overnight, with no activity, the elasticsearch has produced 113GB !!! of logs. crazylog.png

 

Any suggestions of how to disable this (orther than disable the service) ?

 

5 answers

0 vote

Hi Jesper,

 

You might want to rotate the logs and create a cron job to remove/archive old logs every week or months based on your preferences. I've found similar threat below discussing a similar issue whereby logs are piling up  :

https://answers.atlassian.com/questions/323782/jira-log-rotation

https://answers.atlassian.com/questions/200444

Hope that will be able to shed some light.

 

Regards,
Jonathan 

0 vote

Could you please post some details about what is in your logs? The log errors could be because you haven't set up an Elastic search instance. In Bitbucket Server 4.6, the code search feature was released. This required a separate process (elasticsearch) to be started along with Bitbucket Server. Depending on how you installed and started Bitbucket Server, you may already have elasticsearch installed but just haven't started the service yet. If you don't have elasticsearch installed at all, you can take a look at the documentation here to install it. 

Once elasticsearch is configured properly, I would expect the logging to go down. However, without seeing any snippets of your logs there is no way of knowing that for sure.

I restarted the service. After 1/2 hour it hard produced 2 logs, each of 3.8GB.

The huge size of the log is due to an exception which is thrown (and traced) 2.000.000 times in 1/2 hour. See attach images and text.

Remember, similar issue happened when first installed using the onedrive for data path . I then reinstalled it using default path.

--- seach of log file —

atlbitbucket-elasticsearch-x64-stdout.2016-06-19.log(31287550): [bitbucket-search-v1][[bitbucket-search-v1][1]] IndexShardRecoveryException[failed to recovery from gateway]
Found 2032426 occurrence(s) in 1 file(s)

2502016-06-19_200116_one_hour_run.png2016-06-19_200116_one_hour_run_1.png

These 2 exceptions are the ones that fill the log

 

[bitbucket-search-v1][[bitbucket-search-v1][3]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:250)


 at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)


 at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)


 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)


 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)


 at java.lang.Thread.run(Unknown Source)


Caused by: [bitbucket-search-v1][[bitbucket-search-v1][3]] EngineCreationFailureException[failed to create engine]; nested: NoSuchFileException[C:\Atlassian\ApplicationData\Bitbucket\shared\search\data\bitbucket_search\nodes\0\indices\bitbucket-search-v1\3\translog\translog-18.tlog];


 at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:155)


 at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)


 at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1515)


 at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1499)


 at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:972)


 at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:944)
0 vote

This sounds like there is a problem with your search home directory. I would recommend opening a support ticket at http://support.atlassian.com/ so that Atlassian Support is able to do a more thorough investigation.

Suggest an answer

Log in or Sign up to answer
Atlassian Community Anniversary

Happy Anniversary, Atlassian Community!

This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.

Read more
Community showcase
Piotr Plewa
Published Dec 27, 2017 in Bitbucket

Recipe: Deploying AWS Lambda functions with Bitbucket Pipelines

Bitbucket Pipelines helps me manage and automate a number of serverless deployments to AWS Lambda and this is how I do it. I'm building Node.js Lambda functions using node-lambda&nbsp...

2,047 views 1 5
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you