Compressing old logfiles into a single tar archive

This is something that has bothered me. Having tens of thousands of logfiles (especially with backup solutions that are sensitive of the number of files) has become a bit of a problem.

After a bit of googling, I came across this question on stackexchange – full credit to the users there for this tip.

From outside the directory that contains the logfiles:

find /home/user/logs/ -mtime +1 | xargs tar -czvPf /home/user/old_logs_$(date +%F).tar.gz

mtime +1 refers to a file having not been modified for one day. This can of course be changed as your use case requires.

Finally, to delete the files older than one day:

find /home/user/logs/ -mtime +1 -delete

Another useful tip I use to create tar .bz2 files for each year. The logfiles are in format YYYY-, hence the 2012* wildcard.

find * -name "2012*" | tar -cvjSf logs-2012.tar.bz2 -T -

find . -name "2012*" -delete

Leave a Reply

Your email address will not be published. Required fields are marked *