Skip to main content

Maximum Logrotate Compression with bzip2

A Tip for squeezing a little more free space from a VPS

Our web design VPS with Linode has been rock a solid solution over the last 4 years. It currently hosts multiple client sites, freeswitch, mail agent, wireless security camera feeds, and in the past even a Minecraft server and FreeNet; all these services eat up precious hard drive space and with only 20GB the wall can be hit early.

However this measly 20GB does comes with a kick ass IO rate that beats even some dedicated servers

Stop blabering and get to it

Linux logs everything, these logs grow overtime, systems run 'logrotate' to archive and delete old logs - the settings are very liberal and tweaking them could save gigabytes of space, 1.1GB in my instance. Modify your /etc/logrotate.conf as needed but the important bit is compressing by default and using bzip2 instead of gzip to do so.

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep x weeks worth of backlogs - default 4
rotate 2

# Archive log files by adding a YYYYMMDD instead of simply adding a number.
dateext

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# use bzip2 whith higher compression than gzip
compresscmd /bin/bzip2
uncompresscmd /bin/bunzip2
compressoptions -9
compressext .bz2

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

There are also specific application/services log rotation settings in the /etc/logrotate.d directory. Apache by default keeps it's logs for a lengthy 52 weeks! This can be changed by modifying the /etc/logrotate.d/apache2 file as shown below:

/var/log/apache2/*.log {
        weekly
        missingok
        # rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                /etc/init.d/apache2 reload > /dev/null
        endscript
}

Be legendary and do social like this post as that helps me immensely and do follow me on twitter: @danielsokolows.

Comments

  1. Have you ever seen lzip?
    It is available on the epel repository (for Centos & RHEL), and it has much higher compression than bzip2.

    ReplyDelete

Post a Comment

Popular posts from this blog

How to child proof a fireplace

DIY - Do it yourself fireplace child guard Our wonderful 8.5 month old Sofia has become a crawling race car with an untamed thirst for exploration. And so with the cold nights approaching we needed to child proof the fireplace. This however proved to be more difficult than would reasonably expect, I've checked the local Toys "R" Us, Walmart, and even a Canadian Tire with no success for a ready to use product. Internet search was more fruitful and returned a few online stores one could order from, however in all honestly they didn't look too sturdy to me. So I build my own relatively quickly and inexpensively. Materials needed is a privacy plastic lattice - the smallest hole pattern - a few screws and anchors; tools needed are a drill, and a handsaw if you don't have the lattice cut at the store - that’s it. The construction consits of screwing the lattice into the wall and the final product is easiest explained through following pictures. Feel free to +1 this po…

Duplicate value found: duplicates value on record with id: <unknown>.

System.DmlException: Insert failed. First exception on row 0; first error: DUPLICATE_VALUE, duplicate value found: <unknown> duplicates value on record with id: <unknown>.The above error is triggered in the database layer and caused by a trigger or workflow outside of your main code of block that is bubbling this exception. This is rather difficult to track down especially if you are unfamiliar with the code, I am sharing my procedure in the hopes this saves you time - if you find this helpful drop me a line or follow me on twitter @danielsokolows. This error is caused by unique field constraint on the object, so the first step is to examine the object and locate the API names of all unique fieds. You can do this through SF direclty 'Setup < Customize &lt <object being inserted> &lt Fields' or by downloading the `src/objects` metadata information and searching for <unique>; I preffer the latter and actually download ALL matadata information f…

Storing passwords in PuTTY

How to save SSH username/password for auto login in PuTTy The answer is you can't do it...at least in plain PuTTy. However there is an awesome fork with that let's you store the username and password and other additional features called KiTTy.



So grab yourself a copy and +1 this if you do, thanks.