Re: --files-from= and --delete corrent options
On Wed, 2008-09-24 at 15:41 +1000, James Robertson wrote: [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ If /tmp/BACKUPFILE contained a directory e.g. [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE 3/ How would I get it to copy everything under and including /var/data/shares/vmbackups/3/? and delete everything else on the destination? The exclude just needs to be anchored so that it matches only the undesired top-level directories, not files inside the desired top-level directory /var/data/shares/vmbackups/3: rsync -ah --delete --numeric-ids --stats --delete-excluded --include-from=/tmp/BACKUPFILE --exclude='/*' /var/data/shares/vmbackups/ DEST/ The --include-from pattern will match any descendants of /var/data/shares/vmbackups/3 that also happen to be directories named 3, but that should not hurt anything. If you feel like being meticulous, you could anchor that one too: ls -t1 /var/data/shares/vmbackups/ | head -n 1 | sed 's,^,/,' /tmp/BACKUPFILE Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
On Mon, 2008-09-22 at 15:43 +1000, [EMAIL PROTECTED] wrote: I am running the following command to backup a specific file from /tmp/BACKUPFILE which contains the name Friday2008-09-19.bkf. rsync -ah --delete --numeric-ids --stats --delete-excluded --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ The name in /tmp/BACKUPFILE changes each day for example when next run it will be Monday2008-09-19.bkf I use a script before running rsync to obtain the most recently modified file from /var/data/shares/vmbackups/ and cat it into /tmp/BACKUPFILE. I want the /mnt/usb_backup/SFSYDVS01/daily.0/VM/ directory to only ever contain the most recent backup file e.g. [EMAIL PROTECTED]:~# ls -t1 /var/data/shares/vmbackups | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# rsync [CORRECT OPTIONS] --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Friday2008-09-19.bkf Skip to next day. [EMAIL PROTECTED]:~# ls -t1 /var/data/shares/vmbackups | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# rsync [CORRECT OPTIONS] --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Monday2008-09-19.bkf But currently the command grabs everything in /var/data/shares/vmbackups/ and so I end up with: [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Monday2008-09-19.bkf Friday2008-09-19.bkf I expected the --delete option would have removed Friday2008-09-19.bkf in this example (with the --delete option) but it doesn't. I've read the man page and googled a bit but I think i'm getting a bit lost. The current meaning of --files-from is to restrict rsync to processing the paths in the list, and rsync deletes extraneous destination files as part of the processing of the parent directory (assuming -d or -r is on), so you can't delete unlisted files this way. See: https://bugzilla.samba.org/show_bug.cgi?id=3465 Try this instead: rsync [OPTIONS] --include-from=/tmp/BACKUPFILE --exclude='*' /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ This way, rsync will scan the destination directory for deletions (since you have -a without --files-from, which implies -r) and will delete the old files because they are excluded with --delete-excluded. Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
Try this instead: rsync [OPTIONS] --include-from=/tmp/BACKUPFILE --exclude='*' /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ This way, rsync will scan the destination directory for deletions (since you have -a without --files-from, which implies -r) and will delete the old files because they are excluded with --delete-excluded. Matt Thanks Matt, I tried this but it doesn't seem to work for me as you describe. I just created a test setup. The commands I'm running are as follows: [EMAIL PROTECTED]:~# rsync --version rsync version 2.6.9 protocol version 29 [EMAIL PROTECTED]:~# ls /tmp/source/ Thursday2008-09-11.bkf Tuesday2008-09-09.bkf Wednesday2008-09-10.bkf [EMAIL PROTECTED]:~# ls -t1 /tmp/source | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE Thursday2008-09-11.bkf [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ .d..t.. ./ Number of files: 1 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 20 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 42 Total bytes received: 26 sent 42 bytes received 26 bytes 136.00 bytes/sec total size is 0 speedup is 0.00 [EMAIL PROTECTED]:~# ls /tmp/destination/ [EMAIL PROTECTED]:~# -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
On Wed, 2008-09-24 at 13:00 +1000, James Robertson wrote: I tried this but it doesn't seem to work for me as you describe. I just created a test setup. The commands I'm running are as follows: [EMAIL PROTECTED]:~# rsync --version rsync version 2.6.9 protocol version 29 [EMAIL PROTECTED]:~# ls /tmp/source/ Thursday2008-09-11.bkf Tuesday2008-09-09.bkf Wednesday2008-09-10.bkf [EMAIL PROTECTED]:~# ls -t1 /tmp/source | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE Thursday2008-09-11.bkf [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ .d..t.. ./ Number of files: 1 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 20 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 42 Total bytes received: 26 sent 42 bytes received 26 bytes 136.00 bytes/sec total size is 0 speedup is 0.00 [EMAIL PROTECTED]:~# ls /tmp/destination/ [EMAIL PROTECTED]:~# The -include-from=/tmp/BACKUPFILE is short a dash, causing it to be parsed as -i -n -c -l -u -d --rsh=-from=/tmp/BACKUPFILE . Change it to --include-from=/tmp/BACKUPFILE . Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --files-from= and --delete corrent options
Matt McCutchen wrote: On Wed, 2008-09-24 at 13:00 +1000, James Robertson wrote: I tried this but it doesn't seem to work for me as you describe. I just created a test setup. The commands I'm running are as follows: [EMAIL PROTECTED]:~# rsync --version rsync version 2.6.9 protocol version 29 [EMAIL PROTECTED]:~# ls /tmp/source/ Thursday2008-09-11.bkf Tuesday2008-09-09.bkf Wednesday2008-09-10.bkf [EMAIL PROTECTED]:~# ls -t1 /tmp/source | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE Thursday2008-09-11.bkf [EMAIL PROTECTED]:~# rsync -ah --delete --numeric-ids --stats --delete-excluded -include-from=/tmp/BACKUPFILE --exclude='*' /tmp/source/ /tmp/destination/ .d..t.. ./ Number of files: 1 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 20 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 42 Total bytes received: 26 sent 42 bytes received 26 bytes 136.00 bytes/sec total size is 0 speedup is 0.00 [EMAIL PROTECTED]:~# ls /tmp/destination/ [EMAIL PROTECTED]:~# The -include-from=/tmp/BACKUPFILE is short a dash, causing it to be parsed as -i -n -c -l -u -d --rsh=-from=/tmp/BACKUPFILE . Change it to --include-from=/tmp/BACKUPFILE . Matt If /tmp/BACKUPFILE contained a directory e.g. [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE 3/ How would I get it to copy everything under and including /var/data/shares/vmbackups/3/? and delete everything else on the destination? The guys running the backup to /var/data/shares/vmbackups/ are considering copying the files direct rather than sticking it into a *.bkf and have it create folders like 1/, 2/, 3/ with files and folders under that. I came up with running this: [EMAIL PROTECTED]:~# ls /tmp/source/ 1/ 2/ 3/ [EMAIL PROTECTED]:~# ls -t1 /tmp/source/ | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# cat /tmp/BACKUPFILE 3/ [EMAIL PROTECTED]:~# ls -t1 /tmp/source/ | grep -f /tmp/BACKUPFILE -v /tmp/BACKUPEXCLUDE [EMAIL PROTECTED]:~# cat /tmp/BACKUPEXCLUDE 2/ 1/ [EMAIL PROTECTED]:~# rsync -ah --numeric-ids --stats --delete-excluded --include-from=/tmp/BACKUPFILE --exclude-from=/tmp/BACKUPEXCLUDE /tmp/source/ /tmp/destination/ [EMAIL PROTECTED]:~# ls /tmp/destination/ 3/ Which works and copies the subdirectories and files but is there a more elegant solution without the need for the /tmp/BACKUPEXCLUDE file? I think i'll rename /tmp/BACKUPFILE to /tmp/BACKUPDIR :) Apologies as they only just decided this today after I came back with the solution you helped me with. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--files-from= and --delete corrent options
Hi, I am running the following command to backup a specific file from /tmp/BACKUPFILE which contains the name Friday2008-09-19.bkf. rsync -ah --delete --numeric-ids --stats --delete-excluded --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ The name in /tmp/BACKUPFILE changes each day for example when next run it will be Monday2008-09-19.bkf I use a script before running rsync to obtain the most recently modified file from /var/data/shares/vmbackups/ and cat it into /tmp/BACKUPFILE. I want the /mnt/usb_backup/SFSYDVS01/daily.0/VM/ directory to only ever contain the most recent backup file e.g. [EMAIL PROTECTED]:~# ls -t1 /var/data/shares/vmbackups | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# rsync [CORRECT OPTIONS] --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Friday2008-09-19.bkf Skip to next day. [EMAIL PROTECTED]:~# ls -t1 /var/data/shares/vmbackups | head -n 1 /tmp/BACKUPFILE [EMAIL PROTECTED]:~# rsync [CORRECT OPTIONS] --files-from=/tmp/BACKUPFILE /var/data/shares/vmbackups/ /mnt/usb_backup/SFSYDVS01/daily.0/VM/ [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Monday2008-09-19.bkf But currently the command grabs everything in /var/data/shares/vmbackups/ and so I end up with: [EMAIL PROTECTED]:~# ls /mnt/usb_backup/SFSYDVS01/daily.0/VM/ Monday2008-09-19.bkf Friday2008-09-19.bkf I expected the --delete option would have removed Friday2008-09-19.bkf in this example (with the --delete option) but it doesn't. I've read the man page and googled a bit but I think i'm getting a bit lost. Sorry if this seems long winded but I hope you understand what I'm trying to do. Thanks James -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
