How to Use Logrotate for Managing Log Files

Logging is an important for software development and operation. Over the time, the log files can grow fast to fill up the disk space. To avoid the problem, log files are rotated, compressed, and deprecated based on certain rules, for example, periodically, over certain size limit, and retention limit.

Most mordern logging frameworks can do log rotation and compression, but different applications may use different frameworks thus configure them differently. If you want to have a solution across different applications for consistent policies, the logrote (https://fedorahosted.org/logrotate/) is a good choice.

Bothered by SLOW Web UI to manage vSphere? Want to manage ALL your VMware vCenters, AWS, Azure, Openstack, container behind a SINGLE pane of glass? Want to search, analyze, report, visualize VMs, hosts, networks, datastores, events as easily as Google the Web? Find out more about vSearch 3.0: the search engine for all your private and public clouds.

“The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.”

Installation
To install the logrotate, you can download it from the project home, or simply use yum or apt-get:

# sudo apt-get update
# sudo apt-get install logrotate

Configuration Structure
By default, the logrotate runs following the rules defined in the /etc/logrotate.conf as shown below.

# see "man logrotate" for details
# rotate log files weekly
weekly
 
# keep 4 weeks worth of backlogs
rotate 4
 
# create new (empty) log files after rotating old ones
create
 
# uncomment this if you want your log files compressed
#compress
 
# packages drop log rotation information into this directory
include /etc/logrotate.d
 
# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}
 
/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

Who Drives It?
Having the configruation file in place is not enough. It got to be used by the logrotate command, which cannot call itself. To make the command run automatically, the cron job is used. Inside the /etc/cron.daily, there is a file called logrotate defined as follows:

#!/bin/sh
 
/usr/sbin/logrotate /etc/logrotate.conf >/dev/null 2>&1
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

When the daily run comes, the logrotate will read the configuration file and included files and act accordingly.

Add Your Own Rules
Normally you don’t want to change this main configuration file, but to add new configuration under the /etc/logrotate.d directory. The logrotate will include them as the include line dictates.

As a convention, you can create a configuration file in the directory per application so that it’s easy to find it. As you find, the file do not have to be named as *.conf at all.

The following is a quick sample from initial installation.

# cat /etc/logrotate.d/apt
/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}
 
/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

There can be many rules in a configuration file. For each rule, there are two parts:

1. What. It can specify a single file or a group of files using pattern matching. You can have more than one file name or pattern in this part. For example,

/var/log/message /var/log/doublecloud/*.log {
    daily
    rotate 5
}

2. How. It specifies what to do with the log file(s). Most of these actions are pretty self-explanatory. They covers the actions like compress, frequency like daily, number limit like rotate 12, and special cases like missingok.

Testing
Once you write your own configuration, you want to test it out immediately even though the rule says it’s daily or weekly. Luckily there is a command option for this:

# logrotate -d -f /etc/logrotate.d/doubecloud.conf

With the -f switch the action will happen right away regardless of the frequency rule. The -d switch will display more details for debugging purpose. Lastly, you don’t need to use the default configuration file, but your own configuration for the testing. That will speed up the process for quick turn around.

This entry was posted in Applications & Tools, Software Development and tagged , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

2 Comments

  1. Posted October 14, 2014 at 6:38 pm | Permalink

    Simply want to say your article is as amazing. The clarity for your
    post is simply excellent and that i could think you are an expert on this subject.

    Well with your permission let me to snatch your feed to stay updated with impending post.
    Thanks a million and please carry on the gratifying work.

  2. Posted September 6, 2015 at 10:12 pm | Permalink

    Good !|Cool! I love your this bolg.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  • NEED HELP?


    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.