On Fri, Aug 25, 2006 at 09:50:53AM +0100, Hagakure wrote: > Hi, > > I've got to upgrade our backup system (bash script) that currently uses > rsync to backup two sites to a storage server. > > Currently rsync is used to do an incremental copy of the data and then > we archive it using tar. Unfortunately, the archives are now taking up a > huge amount of hdd space. > > What I am looking for is a pointer so that we still use rsync to copy > the data, but rather than archive the entire backup, we have a tar file > for each days differences.
I suggest you look at the --link-dest option of rsync. That tells rsync to reuse identical files from previous backups by hardlinking to them. It really works well, and is a very efficient way of doing on disk backups. Especially if you want to do daily, or even hourly backups. > Ideally, we would be able to insert the differences (file names only!) > into a mysql db so we can recover files quickly. If you use the rsync --link-dest method then it's not really necessary. Each incremental backup is a complete tree that represents a point in time of the backup source. So finding files is dead easy. Just use find or locate. :-) > I looked into various options, but wondered if anyone might have a > pointer to enable us to do this. I've attached one of our backup scripts that uses rsync --link-dest to do incremental local backups. The same concept can be used over the network to. -- CJ van den Berg mailto:[EMAIL PROTECTED] xmpp:[EMAIL PROTECTED]
#!/bin/bash SOURCE=/home/ BACKUPDIR=/backups echo backing up to ${BACKUPDIR} if [ ! -d ${BACKUPDIR} ] then echo ERROR: Backup dir does not exist! exit 1 fi if [ -e ${BACKUPDIR}/0 ] then echo shifting backup dirs... pushd ${BACKUPDIR} > /dev/null for i in *; do mv ${i} $(( ${i} + 100000 + 1 )); done for i in *; do mv ${i} $(( ${i} - 100000 )); done popd > /dev/null fi TARGET=${BACKUPDIR}/0 LINKDEST=--link-dest=../1 echo backing up ${SOURCE}... rsync -a ${LINKDEST} ${SOURCE} ${TARGET} || exit 1 echo syncing... sync echo done.
signature.asc
Description: Digital signature