How to close Logs or delete Logs from GUI or Command line

[willie]

Hi All 

Jira/Confluence uses too many HD space, please help me to share how to close logs or delete logs from GUI or Command line 

[Radek Dostál · Accepted solution]

I really don't think it's the logs that are taking up the space, because they're limited by logger properties. I don't know what you mean by "GUI or command line", it's whichever operating system it's running on. You should be able to know how to work with files, otherwise I don't see how or who installed the apps there in the first place.

[willie]

@Radek Dostál 
Thanks you for your response, I've put more information for my issue under @Nic Brough -Adaptavist-  ANSWER content, please help me to fix this problem. thanks you again.

[Nic Brough -Adaptavist-]

Not every install of Atlassian stuff sets up log clearing in a good way.  I tend to use "logrotate" to remove old logs from <jira install>/log and <jira data>/logs, and I usually set it to 3-4 weeks (I figure if it goes "bang" on day 1 of my 2-week holiday, it'll be useful to have the logs when I get back, but I'll take a copy if I really need to do more work with them, so after a log is 4 weeks old, I'm never going to look at it)

But, as @Radek Dostál says, your disk space probably isn't being chewed up by logs, especially if your Jira was installed in a way that means it automatically removes the old ones.

The places where Atlassian systems tend not to auto clean are actually <home>/temp and <home>/exports - this is where they create temporary files, backups and exports, and unless you've got some pretty specific settings or defaults, Atlassian stuff will flow into those directories and get bigger and bigger until you realise there's a problem.

Again, I tend to use logrotate on those directories, but on non-unix-like servers or ones that don't want to do it that way, I've also implemented simple "delete files over a week old" from those directories, either on Atlassian system (re)start or simply weekly.

My point here is that you should look at what is consuming all your disk space, not just assume your logs are the problem.

[willie]

Hi @Nic Brough -Adaptavist- 

Thanks you very much for your response, let me explain my infrastructure for you. Could you give me some advises, thanks you again . 

1. My OS(ubuntu) only use 32G HD space  / Data HD 2T

2. User accounts 250(frequently users 50~100)

3. 1 VM on AZURE included 4 containers (Nginx / Mysql / JIRA / Confluence)

4.  Jira/Confluence version /  Atlassian JIRA Project Management Software (v7.6.13#76015-sha1:18c5011)/ Atlassian Confluence 6.6.14

5. logs used rate (1M per day) / 3/17 (2.02G) -- 3/18 (1.91G) -- 3/20 (1.89G) -- 3/21 (1.94) -- 3/23 (1.72) -- 3/27 (1.77)

VM HD status

Jira HD status

Confluence HD status

PS:

1. Is it a good method to use the GUI to turn off loggers ??

2. In fact, I've used many times to delete logs file for releasing HD space, but I don't think so it is a good way. 

3. In my opinion, to increase OS HD space maybe it is a great idea but logs issue still doesn't solve, it will be continuing grow up, so what can I do.

[Nic Brough -Adaptavist-]

Turning down the logging is not going to save you much space.

A better thing to do is look at the logs to see what it is logging.  Is it rammed with errors?  If you've got scriptrunner, do your scripts log too much?

2Gb logs every day for a system with that many users is very high, and I suspect your system is logging a lot as a way to ask for help! 

Last time I saw this, it was a larger system, logging 20Gb a day.  A brief read of just the first few hundred lines of one told us they were logging too much in some of their scripts (debugging/testing logging had not been turned off after the script had been tested), they had a misconfigured app and a bunch of errors being repeated because they had misconfigured listeners and workflow functions.

[willie]

@Nic Brough -Adaptavist-  Thank you for your help. As you suggested, could you please provide some guidelines to help me resolve the issue of excessive system logging? I would appreciate any assistance you can offer. thanks again, willie.

[Nic Brough -Adaptavist-]

Start by reading the logs.  

You could run them through log analysis software, or do this "properly" by taking a whole one and trying to group all the different errors together to do some pattern analysis, but a far quicker solution is to just look at them.

Take a chunk off the top of the current log - do something like "tail -5000 atlassian-jira.log > quicklook.txt", then open quicklook.txt in a viewer or editor and scroll down it.

You're looking for patterns in here - are you getting loads of similar informational login lines?  Lots of Java stack-traces?  Lot's of "can't connect to remote system"?

If you can identify something that is happening a lot, then look at what is causing the message to be written into the log.

Edit - I'm sorry, I missed a bit off. 

The first time you have a quick look, you are likely to find some easy wins - if a large number of the logs are a warning from a script, or a connection failure, or a stack trace, for example, then fix the obvious problem(s) first, and repeat the quick-look a couple of days later.  This will reduce the logging volume immediately but also make it a lot easier to identify the next candidates for a fix.

[willie]

Hi @Nic Brough -Adaptavist-  

Thank you very much for your answer, however, could you please give me more support, thanks again.

My question:

Could you tell me where to find the system logs? In a previous message, you mentioned using the command 'tail -5000 "atlassian-jira.log" > quicklook.txt', but I'm not sure which folder to look in. I see access_log files for each day, but I haven't found any error messages in them.

[Nic Brough -Adaptavist-]

The first logs you want to look at will be under <jira home>/logs. 

You may, later, want to check out <jira install>/log, but it's rare that those grow rapidly, and the installers are much better at setting up cleaning for them.   I suspect you'll only need to worry about the ones in "home"