In my last article, I talked about how to use logrotate to manage logs. As everything else, there are some tricks that are only learned when using it. Here are a few tricks and tips I learned recently. Hope it can save you some time.
Stickiness of Logrotate Rules
Bothered by SLOW Web UI to manage vSphere? Want to manage ALL your VMware vCenters, AWS, Azure, Openstack, container behind a SINGLE pane of glass? Want to search, analyze, report, visualize VMs, hosts, networks, datastores, events as easily as Google the Web? Find out more about vSearch 3.0: the search engine for all your private and public clouds.
There was once a very strange behavior with logroate in a project – some of the log files got rotated even though we didn’t have any rule in any configuration files in the /etc/logrotate.conf or /etc/logrotate.d. After studying all of these file, we still could not find any clue. The studying includes searching, analyzing the matching patterns.
It turned out that a rarely used script called logrotate command with a configuraton file elsewehre, and the rules there got into logrotate cache file: /var/lib/logrotate/status. Once the rules get in, they stay there forever.
Since we don’t want it to be there in the first place, we change the logroate command to use a temporary status file just for the configuration used in the rare command.
# logrotate -s /tmp/logstatus /usr/share/doublecloud/doublecloud.conf
Timing of Compress and Postrotate Script
While using the logroate, I wanted to move the compressed log files to another directory. So I wrote a short script for that. Then I got error complaining that compress cannot find the files. It turned out that the rotated files had been moved away before compression happened, thus could not be found by the compress.
There isn’t much we can do with the order of actions because that is defined by logrotate. To work around this, just move my script away after calling logrotate.
notifempty and copytruncate
If you have these two actions in one rule, you may see the *.log file is sized 0 and the other *.log.1 file keeps growing in size. The *.log.1 file is the active one, but *.log is not. This is OK as long as you know what log file to look for. After all, there is no data loss. In practice, it’s not convenient because most of us will simply assume *.log is the active one.
It turns out that logrotate renames the *.log to be *.log.1, but the application that writes to it still hold handle and continue to write to it. It depends on the applcations. Some may open the *.log based on the name, so it may not be a problem there.
In theory, the notifempty and copytruncate should not be in conflict. But in reality, it seems so. If notifempty is removed, the copytruncate starts to work as expected, which is to copy the *.log to *.log.1.