Wednesday, September 19, 2012

Maximum Logrotate Compression with bzip2

A Tip for squeezing a little more free space from a VPS

Our web design VPS with Linode has been rock a solid solution over the last 4 years. It currently hosts multiple client sites, freeswitch, mail agent, wireless security camera feeds, and in the past even a Minecraft server and FreeNet; all these services eat up precious hard drive space and with only 20GB the wall can be hit early.

However this measly 20GB does comes with a kick ass IO rate that beats even some dedicated servers

Stop blabering and get to it

Linux logs everything, these logs grow overtime, systems run 'logrotate' to archive and delete old logs - the settings are very liberal and tweaking them could save gigabytes of space, 1.1GB in my instance. Modify your /etc/logrotate.conf as needed but the important bit is compressing by default and using bzip2 instead of gzip to do so.

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep x weeks worth of backlogs - default 4
rotate 2

# Archive log files by adding a YYYYMMDD instead of simply adding a number.
dateext

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# use bzip2 whith higher compression than gzip
compresscmd /bin/bzip2
uncompresscmd /bin/bunzip2
compressoptions -9
compressext .bz2

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

There are also specific application/services log rotation settings in the /etc/logrotate.d directory. Apache by default keeps it's logs for a lengthy 52 weeks! This can be changed by modifying the /etc/logrotate.d/apache2 file as shown below:

/var/log/apache2/*.log {
        weekly
        missingok
        # rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                /etc/init.d/apache2 reload > /dev/null
        endscript
}

Be legendary and do social like this post as that helps me immensely and do follow me on twitter: @danielsokolows.

2 comments:

  1. Have you ever seen lzip?
    It is available on the epel repository (for Centos & RHEL), and it has much higher compression than bzip2.

    ReplyDelete