Home > Applications & Tools, Software Development > How to Use Logrotate for Managing Log Files

How to Use Logrotate for Managing Log Files

August 8th, 2014 Leave a comment Go to comments

Logging is an important for software development and operation. Over the time, the log files can grow fast to fill up the disk space. To avoid the problem, log files are rotated, compressed, and deprecated based on certain rules, for example, periodically, over certain size limit, and retention limit.

Most mordern logging frameworks can do log rotation and compression, but different applications may use different frameworks thus configure them differently. If you want to have a solution across different applications for consistent policies, the logrote (https://fedorahosted.org/logrotate/) is a good choice.

Lost VMs or Containers? Too Many Consoles? Too Slow GUI? Time to learn how to "Google" and manage your VMware and clouds in a fast and secure HTML5 App.

“The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.”

To install the logrotate, you can download it from the project home, or simply use yum or apt-get:

# sudo apt-get update
# sudo apt-get install logrotate

Configuration Structure
By default, the logrotate runs following the rules defined in the /etc/logrotate.conf as shown below.

# see "man logrotate" for details
# rotate log files weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
# uncomment this if you want your log files compressed
# packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    create 0664 root utmp
    rotate 1
/var/log/btmp {
    create 0660 root utmp
    rotate 1

Who Drives It?
Having the configruation file in place is not enough. It got to be used by the logrotate command, which cannot call itself. To make the command run automatically, the cron job is used. Inside the /etc/cron.daily, there is a file called logrotate defined as follows:

/usr/sbin/logrotate /etc/logrotate.conf >/dev/null 2>&1
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
exit 0

When the daily run comes, the logrotate will read the configuration file and included files and act accordingly.

Add Your Own Rules
Normally you don’t want to change this main configuration file, but to add new configuration under the /etc/logrotate.d directory. The logrotate will include them as the include line dictates.

As a convention, you can create a configuration file in the directory per application so that it’s easy to find it. As you find, the file do not have to be named as *.conf at all.

The following is a quick sample from initial installation.

# cat /etc/logrotate.d/apt
/var/log/apt/term.log {
  rotate 12
/var/log/apt/history.log {
  rotate 12

There can be many rules in a configuration file. For each rule, there are two parts:

1. What. It can specify a single file or a group of files using pattern matching. You can have more than one file name or pattern in this part. For example,

/var/log/message /var/log/doublecloud/*.log {
    rotate 5

2. How. It specifies what to do with the log file(s). Most of these actions are pretty self-explanatory. They covers the actions like compress, frequency like daily, number limit like rotate 12, and special cases like missingok.

Once you write your own configuration, you want to test it out immediately even though the rule says it’s daily or weekly. Luckily there is a command option for this:

# logrotate -d -f /etc/logrotate.d/doubecloud.conf

With the -f switch the action will happen right away regardless of the frequency rule. The -d switch will display more details for debugging purpose. Lastly, you don’t need to use the default configuration file, but your own configuration for the testing. That will speed up the process for quick turn around.

  1. October 14th, 2014 at 18:38 | #1

    Simply want to say your article is as amazing. The clarity for your
    post is simply excellent and that i could think you are an expert on this subject.

    Well with your permission let me to snatch your feed to stay updated with impending post.
    Thanks a million and please carry on the gratifying work.

  2. September 6th, 2015 at 22:12 | #2

    Good !|Cool! I love your this bolg.

  1. No trackbacks yet.