Automate cleanup of JIRA logs

Kota Sreenivasa Shravana Kumar August 12, 2013

How to automate the clean up of JIRA logs (/var/atlassian/application-data/jira/log and /opt/atlassian/jira/logs ) ?

- jira logs

- tomcat logs

- old backup files

If someone is already having script somewhere, please share. Thanks

2 answers

1 accepted

4 votes
Answer accepted
Robert MOTA August 12, 2013

Hi,

Two things are to do (description below on Unix/Linux) :

To rotate tomcat logs

To purge files older than x days

1) To rotate tomcat logs

- Put this file (attached) log.rotate (c) Copyright 2004 Steven P Kneizys into the logs directory of your java application instance under Tomcat. This script is in Perl (verify that /usr/bin/perl is well the path of Perl language on the server)

- Set the file executable (chmod +x command line for example)

- Modify le file catalina.sh of your Tomcat distribution directory /opt/app/<tomcat-distrib>/bin/catalina.sh replacing 2 time the line (for Tomcat 6.0.x or higher)

&gt;&gt; "$CATALINA_BASE"/logs/catalina.out 2&gt;&amp;1 &amp;

by these 2 lines (no space after "\")

2&gt;&amp;1 | "$CATALINA_BASE"/logs/rmo.log.rotate \
      "$CATALINA_BASE"/logs/catalina.out.%Y-%M-%D.log &amp;

- Stop the application ./run.sh stop <instance-name> (verify that the process is well stopped, otherwise kill it)

- Remove the old file catalina.out

- Start the application ./run.sh start <instance-name>

- Tomcat create again an empty catalina.out file on every start, and this file is not touch after. Its modification date is the date of the last start of tomcat. The rotated log files look like catalina.out_log.2013-08-13.txt

Info : on standalone installations, even if a rotation is in place for atlassian logs, catalina.out is filled. To activate the rotation, modify the file /opt/app/atlassian-standalone-appli/apache-tomcat/bin/catalina.sh

2) To purge files older than x days

- Put this file (attached) log.purge (c) Copyright 2004 Steven P Kneizys into the logs or save directory to purge. This script is in Perl (verify that /usr/bin/perl is well the path of Perl language on the server)

- Set the file executable (chmod +x command line for example)

- To manually purge the logs more than 10 days, into to logs directory, you can use the command line below
./log.purge "." 10 "(catalina.2)(.)*(.log$)"
- To automate it, set a daily cron entry at 1:05am, for instance
Login as <tt>root</tt>
Edit the crontab file crontab -e
Add the line below (set your proper path and file name regex of logs or files to purge)
5 1 * * * su - app -c  'cd /opt/app/tomcat/jira/logs; ./rmo.log.purge "." 10 "(catalina.out.2)(.)*(.log$)" &gt;/dev/null 2&gt;&amp;1'
Save it
- A cron line must be created for every directory where a log.purge file must run
- Example for Crowd standalone
5 1 * * * su - app -c  'cd /opt/app/atlassian-crowd-2.3.4/apache-tomcat/logs; \
      ./rmo.log.purge "." 10 "(catalina.2)(.)*(.log$)" &gt;/dev/null 2&gt;&amp;1'
5 1 * * * su - app -c  'cd /opt/app/atlassian-crowd-2.3.4/apache-tomcat/logs; \
      ./rmo.log.purge "." 10 "(manager.2)(.)*(.log$)" &gt;/dev/null 2&gt;&amp;1'
5 1 * * * su - app -c  'cd /opt/app/atlassian-crowd-2.3.4/apache-tomcat/logs; \
      ./rmo.log.purge "." 10 "(localhost.2)(.)*(.log$)" &gt;/dev/null 2&gt;&amp;1'

Hope that will help you, best regards.

log.rotate

#!/usr/bin/perl
# (be sure to set this to your perl path and 'chmod +x' this script.)

# spk.log.rotate script for piped logging
#
# Copyright 2004 Steven P. Kneizys
#
# sample usage for piping from program "program":
#
#  ... program | spk.log.rotate /path/to/logs/%Y-%M-%D-logname_log
#

$logname_template = $ARGV[0];
$file = "";

while(&lt;STDIN&gt;) {
  $line = $_;
  $time = time;
  if (($file eq '') || ($time gt (60 + $last_time))) {
    chomp($mydate=`date +%Y-%m-%d`);
    $last_time = $time;
    (@date)=split(/-/,$mydate);
    $temp = $logname_template;
    $temp =~ s/%Y/$date[0]/gi;
    $temp =~ s/%M/$date[1]/gi;
    $temp =~ s/%D/$date[2]/gi;
  }
  if ($file ne $temp) {
    if ($file ne '') {close MYFILE;}
    $file = $temp;
    open MYFILE,"&gt;&gt;$file";
    select((select(MYFILE), $| = 1)[0]); # autoflush
  }
  print MYFILE $line;
}
if ($file ne '') {close MYFILE;}

log.purge

#!/usr/bin/perl
# (be sure to set this to your perl path and 'chmod +x' this script.)

#
# Script name: spk.log.purge
# Purge files in a directory (not a directory tree!) based on age.
# Copyright 2004 Steven P Kneizys
#

if (@ARGV &lt; 3) { #Requires three input parameters, fourth is optional
        print "USAGE: perl spk.log.purge DirectoryPath MaxAge \"pattern\" [test]\n";
        print "       DirectoryPath = Directory to purge from.\n";
        print "       MaxAge        = days of keep files \n";
        print "       \"pattern\"     = pattern to match for file names\n";
        print "       [test]        = optional parameter to print instead of delete.\n";
        exit 1;
   }

$dir = $ARGV[0];
$maxage = $ARGV[1];
$pattern = $ARGV[2];

opendir(DIR, $dir);
foreach (readdir(DIR)) {
  $path = "$dir/$_";
  if ((-f $path) &amp;&amp; (-M $path &gt; $maxage) &amp;&amp; ($_ =~ $pattern)) {
    if ($ARGV[3] =~ /test/i) {
      print `ls -al $path`;
    } else {
      unlink $path; # delete the file
    }
  }
}
closedir(DIR);
Dave C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
October 12, 2013
5 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
August 12, 2013
Most of us simply use logrotate on Unix boxes for the logs. For the backups, I tend to simply use a one line "find files over x days old -exec rm {} \;" in cron .
KAGITHALA BABU ANVESH
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 3, 2022

Hi @Nic Brough -Adaptavist- ,

Thanks for the reply, I'm a bit new to linux. 

due to large number of old logs, our disk space is getting low, we are looking for the same kind of solution, 

If possible can you please more elaborate on the Cron expression,

 

Thanks,

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 3, 2022

What do you need in "elaboration"?  

KAGITHALA BABU ANVESH
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 3, 2022

Cron expression to delete the files on automaton based number of days older

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
March 3, 2022

Don't ever do things like messing with Cron unless you fully understand what they are doing.  You should read a tutorial or docs on Cron (and "find" if you're going to be using a command like the one I gave earlier)

KAGITHALA BABU ANVESH
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
March 3, 2022

Sure, Thanks

Suggest an answer

Log in or Sign up to answer