Crontab doesn't complete a script - was Re: Longstanding rsync script no longer excludes
On 17/09/16 17:12, Clive Menzies wrote: WORKING version: # Good rsync options for uhuru_backups. rsync_opts="-av --exclude-from=exclude_list --delete --delete-excluded" The exclude_list is a file with one excluded directory per line with an asterisk at the end directory name. $rsync $rsync_opts $password Uhuru::rsync \ /root/uhuru_backup/uhuru/home/ > /var/rsync/uhuru.$NOW.log The daily backup script works fine when run manually as sudo but doesn't complete when run as a root cronjob. I know the script starts because it creates a log. When successful it gives a list of files copied but when run as a cronjob it's blank (0 bytes) $ sudo crontab -l 20 21 1 * * /root/uhuru_backup/uhuru.monthly.sh > /dev/null 25 21 * * 7 /root/uhuru_backup/uhuru.weekly.sh > /dev/null 30 21 * * * /root/uhuru_backup/uhuru.daily.sh > /dev/null This should be impossible but I've got the same issue on two backup servers. Regards Clive -- Clive Menzies http://freecriticalthinking.org
Re: Longstanding rsync script no longer excludes - SOLVED
On 17/09/16 13:32, Mike Bird wrote $rsync $rsync_opts $password FileServer::rsync \ /home/backup/home/ > /var/rsync/FileServer.$NOW.log I don't see where the $excludes you built is actually used. On 17/09/16 13:44, Clive Menzies wrote: The script wasn't changed and I've always assumed the $rsync-opts took care of this. The script was copied from someone else and adapted. # Good rsync options for FileServer_backups. rsync_opts="-av --delete --delete-excluded" Thanks to prompting by Mike, I've changed the way excludes are handled and it works :-) OLD version: # Good rsync options for uhuru_backups. rsync_opts="-av --delete --delete-excluded" # A list of files and directories that do not need to be backed up exclude_list="music funstuff cma/TransitStuff lost+found/" excludes="" for exclude in $exclude_list; do excludes="$excludes --exclude=$exclude" WORKING version: # Good rsync options for uhuru_backups. rsync_opts="-av --exclude-from=exclude_list --delete --delete-excluded" The exclude_list is a file with one excluded directory per line with an asterisk at the end directory name. Regards Clive -- Clive Menzies http://freecriticalthinking.org
Re: Longstanding rsync script no longer excludes
On 17-09-2016 07:42, Clive Menzies wrote: > > Hi > > One of the issues we've encountered following a recent Jessie upgrade, > (around 5th September) is an rsync script seems to have changed behaviour. > > We've been using rsync for a daily automated incremental backup and > have three scripts to maintain an archive of daily snapshots for a > week; weekly snapshots for a month; and 6 monthly snapshots (files are > irretrievable 6months after they been deleted from the backed up server). > > We've two pairs of FileServer and BackupServer, rsyncd runs on > FileServer and the following daily script on BackupServer > > #!/bin/bash > > # This script is called daily from cron to perform overnight backups > # The full paths of the programs used in this script > rm=/bin/rm > mv=/bin/mv > cp=/bin/cp > rsync=/usr/bin/rsync > > # Good rsync options for FileServer_backups. > rsync_opts="-av --delete --delete-excluded" > > # The name of the file containing the rsync connection password > password="--password-file=/etc/.rs_pass" > > # A list of files and directories that do not need to be backed up > exclude_list="music funstuff cma/TransitStuff lost+found/" > excludes="" > for exclude in $exclude_list; do > excludes="$excludes --exclude=$exclude" > done > > # define NOW for adding date to rsync log filename eg. FileServer.date > NOW=$(date +"%F.%H:%M") > > # Move all other backups up a level. Copy previous backup to > /backup/daily. > # Backup FileServer according to the [rsync] sections of the > rsyncd.conf files > # on FileServer. Use the password given in /etc/.rs_pass. Dump any > output and > # error messages to /var/rsync/FileServer > mv /home/backup/snapshot/day.5 /home/backup/snapshot/day.6 > mv /home/backup/snapshot/day.4 /home/backup/snapshot/day.5 > mv /home/backup/snapshot/day.3 /home/backup/snapshot/day.4 > mv /home/backup/snapshot/day.2 /home/backup/snapshot/day.3 > mv /home/backup/snapshot/day.1 /home/backup/snapshot/day.2 > cp -al /home/backup/home /home/backup/snapshot/day.1 > $rsync $rsync_opts $password FileServer::rsync \ > /home/backup/home/ > /var/rsync/FileServer.$NOW.log > > The incremental backups have worked seamlessly for over 10 years but > recently, possibly related to the upgrade of Debian Jessie, the > exclude options have ceased to have effect. > > I've dug around the docs but haven't found anything relevant. Any > suggestions as to what may be wrong? > You define $excludes, but you don't seem to use it anywhere. BTW, your rotating backups are basically what rsnapshot does, you might want to look at that. -- Eduardo M KALINOWSKI edua...@kalinowski.com.br
Longstanding rsync script no longer excludes
Hi One of the issues we've encountered following a recent Jessie upgrade, (around 5th September) is an rsync script seems to have changed behaviour. We've been using rsync for a daily automated incremental backup and have three scripts to maintain an archive of daily snapshots for a week; weekly snapshots for a month; and 6 monthly snapshots (files are irretrievable 6months after they been deleted from the backed up server). We've two pairs of FileServer and BackupServer, rsyncd runs on FileServer and the following daily script on BackupServer #!/bin/bash # This script is called daily from cron to perform overnight backups # The full paths of the programs used in this script rm=/bin/rm mv=/bin/mv cp=/bin/cp rsync=/usr/bin/rsync # Good rsync options for FileServer_backups. rsync_opts="-av --delete --delete-excluded" # The name of the file containing the rsync connection password password="--password-file=/etc/.rs_pass" # A list of files and directories that do not need to be backed up exclude_list="music funstuff cma/TransitStuff lost+found/" excludes="" for exclude in $exclude_list; do excludes="$excludes --exclude=$exclude" done # define NOW for adding date to rsync log filename eg. FileServer.date NOW=$(date +"%F.%H:%M") # Move all other backups up a level. Copy previous backup to /backup/daily. # Backup FileServer according to the [rsync] sections of the rsyncd.conf files # on FileServer. Use the password given in /etc/.rs_pass. Dump any output and # error messages to /var/rsync/FileServer mv /home/backup/snapshot/day.5 /home/backup/snapshot/day.6 mv /home/backup/snapshot/day.4 /home/backup/snapshot/day.5 mv /home/backup/snapshot/day.3 /home/backup/snapshot/day.4 mv /home/backup/snapshot/day.2 /home/backup/snapshot/day.3 mv /home/backup/snapshot/day.1 /home/backup/snapshot/day.2 cp -al /home/backup/home /home/backup/snapshot/day.1 $rsync $rsync_opts $password FileServer::rsync \ /home/backup/home/ > /var/rsync/FileServer.$NOW.log The incremental backups have worked seamlessly for over 10 years but recently, possibly related to the upgrade of Debian Jessie, the exclude options have ceased to have effect. I've dug around the docs but haven't found anything relevant. Any suggestions as to what may be wrong? Thanks Clive -- Clive Menzies http://freecriticalthinking.org
