Rotating Log Files with Logback

Just fixed an issue related to configuration of Logback recently. You may be wondering why the Logback is used given that it’s much less known than Log4j and Java Logging. Very good question. This page from Logback may provide you some insights. I haven’t tested the performance, but it’s said to be 10 times faster than others. There is also an independent version of comparison on StackOverflow. After browsing it, I didn’t have an impression that I have to use one over the other.

Anyway. Just like other growing log files, we want the Logback logs to rotate over certain size and limit the total number of files. In that way, we can control the total disk space used for the log files – it could be a disaster if the log files fill all the disk.

Bothered by SLOW Web UI to manage vSphere? Want to manage ALL your VMware vCenters, AWS, Azure, Openstack, container behind a SINGLE pane of glass? Want to search, analyze, report, visualize VMs, hosts, networks, datastores, events as easily as Google the Web? Find out more about vSearch 3.0: the search engine for all your private and public clouds.

Of course, there are other ways to achieve this, for example, the logrotate as I blogged before, which can take care of different logs from all sorts of sources. The downside is that it’s tied to the Linux OS and is not as portable as pure Java version like Log4j and Logback.

To make the log files rotate, a rolling file appender should be used. The following configuration, however, did not get what we expected.

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>logFile.%d{yyyy-MM-dd}.log.zip</fileNamePattern>
      <maxHistory>10</maxHistory>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>10MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>

It turned out there are more than one type of policy. Although there is number 10 as maxHistory, it really means 10 days. If you have more than one rotation each day because more than 10MB logs get created, the total number of files exceeds 10, and the exact number is not very predictable.

To be exact on the total log files, FixedWindowRollingPolicy should be used instead. The following configuration will get what’s expected. Note that the historial log file names are different from the first version.

 <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>tests.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>10</maxIndex>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>10MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
</xml>

Having the total size fully in control, we may have another issue – if there are too many log entries written in short time, the history can be quickly overwritten. In other words, the history is out of control when size in control. In general, we care more about size control than history control so the change works for the project.

This entry was posted in Software Development and tagged , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

One Comment

  1. SRD
    Posted September 7, 2016 at 9:09 am | Permalink

    Hi

    Im trying to create log file for every 5 minutes and max history of 10 files, how can i achieve this?

    Since %d{yyyyMMdd-HH-mm} pattern rolls log file every minute, i have overloaded rollover method by extending TimeBasedRollingPolicy class. I could roll log for every 5 minutes, but maxHistory is not working in this case.

    I would be glad if any one cane help me in solve this issue

    Thanks in advance

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  • NEED HELP?


    My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

    Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.