I’ve now got backup scripts happily creating copies of all my subversion repositories and MySQL databases every 24 hours. This is great but it means you end up with an awful lot of backups. I realy don’t need a backup from every day going back forever. But it is nice to have snapshots going back into history in case something subtle has gone wrong.

What I’d really like is to copy the oldest backups into a different directory every seven days, and delete all the backups in the main directory that are older than seven days. Of course I’ll then end up with piles of backups building up in the weekly directory. So I’d like to go through the weekly directory every month, copy the oldest backups into another directory and delete all the weekly backups that are more than one month old.

To do this I give you the snappily named rotate-backups

rotate-backup            rotates backups in a given directory weekly,or monthly

-b                       directory to rotate backups in
-f                       file containing list of backup dirs
-t                       time period (weekly,monthly)

The config file is just a new-line separated list of directories. To make it work I put a script in /etc/cron.weekly like:

rotate-backups -t weekly -f /etc/rotate-backups

and one in cron.monthly:

rotate-backups -t monthly -f /etc/rotate-backups

The backup script makes an assumption that backups created on the same day are from the same backup run. It copies the ‘oldest’ backups by copying files from the same day as the oldest file in the directory. This way it doesn’t have to know anything about what you are backing up or what your naming conventions are.

Also it culls old backups relative to the date of the latest file in the directory. This means that if you stop taking backups the script won’t keep deleting files until you have none left.

We run a subversion service for a number of different research groups. Generally we create a separate subversion repository for each group. Obviously looking after this data is important. It’s not a good idea to lose months of someone’s work.

Fortunately backing up a subversion repository is pretty simple. Subversion ships with a utility called svnadmin. One of the functions of which is to dump a repository to a file.

As the repository owner do:

svnadmin dump /path/to/repository > subversion_dump_file

I have a directory full of subversion repositories so what I really want is a script that will find all the subversion repositories in the directory and dump them with sensible filenames. With my usual lack of imagination I’ve called this script svn_backup. It runs svnadmin verify against each file in the directory it’s given. If any of them turn out to be subversion repositories it dumps them using svnadmin dump.

$ ./svn_backup
svn_backup  -s subversion_dir [-b backup_dir] [ -o logfile]
        script to backup subversion repositories

        -s      directory containging subversion repositories
        -b      directory to dump backups to. defaults to /tmp
        -o      file to output names of backups to

So I now have an entry in cron.daily like:

svn_backup -s /var/www/subversion -o /tmp/svn.log -b /var/backups/svn

The reason I write the backups to a log file is that it allows me to run a script once a week that copies the latest backups to Amazon’s S3 storage system.

The scripts:
svn_backup
s3svnbackup.py