SOLVED: Re: include/exclude: I miss the point
Hi Michal, (hope this mail is short enough for the list, previous one bounced: too big) Read, re-read, read again both your mail and the rsync-documentation (again) and think I get it now. First, regarding your comments (not all): 1. you are right, the .kabc\-files were missing (did not check it properly). Now I have them (see my solution below) rest of your comments let me understand what I read in the manual. One remark: the Exclude .mybackup/backup*.log.* is not redundant, due to the Include .mybackup. The .mozilla/extensions was redundant. 2 Excluding the [A-Z] and [a-z] works when specifying this: hans/[A-Z]* etc. Now I have the following which works perfect for me (test -version): rsync command: rsync -n -v -axhhX --no-perms --omit-dir-times --stats\ --backup --backup-dir=`date +%Y-%m-%d`/updated --suffix=.`date +%H%M`\ --delete --delete-excluded\ --backup-dir-dels=`date +%Y-%m-%d`/deleted --suffix-dels=.`date +%H%M`\ --log-file=/home/hans/.mybackup/backup-test.log.`date +%Y%m%d`-`date +%H%M`\ --include-from=/home/hans/.mybackup/include.txt\ --exclude-from=/home/hans/.mybackup/exclude.txt\ /home/hans /mnt/backup/test/ I herewith attach both the include.txt and exclude .txt and the listing of backed-up files. Best regards, Hans Troost # # to get KAdressbook entries + distribution lists # .kde4/ .kde4/share/ .kde4/share/apps/ .kde4/share/apps/kabc/ .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf # # to get firefox bookmarks # .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* # # to get the backup-scripts and associated files like this # .mybackup .* .mybackup/backup*.log.* .mozilla/* .mozilla/firefox/* .mozilla/firefox/plp6wzjm.default/* .kde4/* .kde4/share/* .kde4/share/apps/* .kde4/share/apps/kabc/* *.*~ Desktop Downloads Movies Music Pictures tmp Videos # # for testing purposes only will be commented out on real use # hans/[a-z]* # hans/[A-Z]* # hans/2*/mnt/backup/test: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 6 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 hans/ /mnt/backup/test/hans: total 0 drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 .kde4/ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 .mozilla/ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 .mybackup/ /mnt/backup/test/hans/.kde4: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 share/ /mnt/backup/test/hans/.kde4/share: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 apps/ /mnt/backup/test/hans/.kde4/share/apps: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 kabc/ /mnt/backup/test/hans/.kde4/share/apps/kabc: total 28 drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 202 2010-09-11 11:55 distlists* -rwxr-xr-x 1 hans hans 21968 2010-09-11 11:55 std.vcf* /mnt/backup/test/hans/.mozilla: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 firefox/ /mnt/backup/test/hans/.mozilla/firefox: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 plp6wzjm.default/ /mnt/backup/test/hans/.mozilla/firefox/plp6wzjm.default: total 0 drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 bookmarkbackups/ /mnt/backup/test/hans/.mozilla/firefox/plp6wzjm.default/bookmarkbackups: total 216 drwxr-xr-x 1 hans hans 0 2010-09-11 21:08 ./ drwxr-xr-x 2 hans hans 0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 33921 2010-09-05 19:16 bookmarks-2010-09-05.json* -rwxr-xr-x 1 hans hans 33921 2010-09-06 20:08 bookmarks-2010-09-06.json* -rwxr-xr-x 1 hans hans 34164 2010-09-07 21:25 bookmarks-2010-09-07.json* -rwxr-xr-x 1 hans hans 34164 2010-09-08 23:14 bookmarks-2010-09-08.json* -rwxr-xr-x 1 hans hans 34164 2010-09-10 14:30 bookmarks-2010-09-10.json* -rwxr-xr-x 1 hans hans 34164 2010-09-11 20:42 bookmarks-2010-09-11.json* /mnt/backup/test/hans/.mybackup: total 64 drwxr-xr-x 1 hans hans0 2010-09-11 21:08 ./ drwxr-xr-x 4 hans hans0 2010-09-11 21:08 ../ -rwxr-xr-x 1 hans hans 471 2010-09-10 20:44 backup-dry-run.sh* -rwxr-xr-x 1 hans hans 1305 2010-09-08 17:06 backup-old.sh* -rwxr-xr-x 1 hans hans 749 2010-09-10 19:43 backup.sh* -rwxr-xr-x 1 hans hans 457 2010-09-11 20:54 backup-test.sh* -rwxr-xr-x 1 hans hans 101 2010-09-10 20:48 exclude-old .txt*
include/exclude: I miss the point
Hi all, I thought I understand how include/exclude and their precedences work (I read the user guide about this), but I seem to miss something. I have 2 questions about this, separated by == Question 1: -- I want to exclude the hidden directories except some sub-dirs or files: .mybackup and its files) and the mozilla firefox bookmarks folder and its contents (.mozilla/firefox/plp6wzjm.default/bookmarkbackups/*) So my exclude-files contains: .* .mybackup/backup.log.* .mozilla/extensions *.*~ Desktop Downloads Movies Music Picturestmp Videos the include file contains: .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* .mybackup Result: 1. Success: with both the .mybackup and the 2 .kde4/share/apps/kabc-files 2. Success: indeed I have my bookmarks in the ../bookmarkbackups/ folder 3. Failure: I have all other files, sub-dirs and their content as well in .mozilla/firefox/plp6wzjm.default. 4. when adding .mozilla/firefox/plp6wzjm.default/Cache .mozilla/firefox/plp6wzjm.default/chrome .mozilla/firefox/plp6wzjm.default/extensions .mozilla/firefox/plp6wzjm.default/firebug .mozilla/firefox/plp6wzjm.default/OfflineCache to the exclude file this folders are not in, but the normal .mozilla/firefox/plp6wzjm.default/ files are. Question: it seems that I miss the point how to do this. can you please give a hint about what I ḿ doing wrong? rsync command )some date-time variable for files, but that is working): rsync -axhhX --no-perms --omit-dir-times\ --backup --backup-dir=$NiceDate/updated --suffix=.$Now\ --delete --delete-excluded\ --backup-dir-dels=$NiceDate/deleted --suffix-dels=.`date +%H%M`\ --log-file=/home/hans/.mybackup/backup.log.$RawDate-$Now\ --include-from=/home/hans/.mybackup/include.txt\ --exclude-from=/home/hans/.mybackup/exclude.txt\ /home/hans /mnt/backup === 2nd, similar question: to play around with this stuff I wanted to test with the hidden files only (.*), so I tried to add all non-hidden directories and normal files to this exclude-test.txt, but did not get it working (it worked too much): exclude-test.txt .* .mybackup/backup.log.* [a-z]* [A-Z]* while my include-test is: .kde4/share/apps/kabc/distlists .kde4/share/apps/kabc/std.vcf .mozilla/ .mozilla/firefox/ .mozilla/firefox/plp6wzjm.default/ .mozilla/firefox/plp6wzjm.default/bookmarkbackups/* .mybackup In this case I don't get any files backed up (dry-run, so checked in log-file only). same rsync command (other log-file and destination) === Thanks for any help, regards, Hans Troost -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: include/exclude: I miss the point
On 10-09-10 21:13, Hans Troost wrote:
So my exclude-files contains:
.*
the include file contains:
.kde4/share/apps/kabc/distlists
.kde4/share/apps/kabc/std.vcf
Result:
1. Success: 2 .kde4/share/apps/kabc-files
Are you sure about that part ? '.*' ('H, .*' as seen by sending rsync)
should short-circuit any .kde/ directory subtree and exclude it from
file list/transfer. You would need something more elaborate to get just
those two fles from .kde4 and nothing else.
Anyway, putting that detail aside and if I'm reading things right.
Your rules (combined) look like:
+, .mozilla/
+, .mozilla/firefox/
+, .mozilla/firefox/plp6wzjm.default/
+, .mozilla/firefox/plp6wzjm.default/bookmarkbackups/*
+, .mybackup
H, .*
H, .mybackup/backup.log.*
H, .mozilla/extensions
H, *.*~
H, Desktop
H, Downloads
H, Movies
H, Music
H, Picturestmp
H, Videos
+, * (implicit final rule)
H - because you added --delete-excluded, so receiving side will not
protect them from deletion.
For receiving rsync that means just:
R, * (as consider everything not part of the transfer a candidate to
delete)
For sending rsync:
S, .mozilla/
S, .mozilla/firefox/
S, .mozilla/firefox/plp6wzjm.default/
S, .mozilla/firefox/plp6wzjm.default/bookmarkbackups/*
S, .mybackup
H, .* - this makes following 2 rules redundant
H, .mybackup/backup.log.*
H, .mozilla/extensions
H, *.*~
H, Desktop
H, Downloads
H, Movies
H, Music
H, Picturestmp
H, Videos
S, * (implicit final rule)
Which does exactly what you experienced:
3. Failure: I have all other files, sub-dirs and their content as well in
.mozilla/firefox/plp6wzjm.default.
They were all caught by the final implicit include, and none of them
matched your H (exclude) rules.
4. when adding
.mozilla/firefox/plp6wzjm.default/Cache
.mozilla/firefox/plp6wzjm.default/chrome
.mozilla/firefox/plp6wzjm.default/extensions
.mozilla/firefox/plp6wzjm.default/firebug
.mozilla/firefox/plp6wzjm.default/OfflineCache
to the exclude file this folders are not in, but the normal
.mozilla/firefox/plp6wzjm.default/ files are.
Being part of exclude rules, they will be catched and excluded from
transfer, but the rest will not.
Note, that nothing from .mybackup that matches H (exclude) rules will be
backed up.
Question: it seems that I miss the point how to do this. can you please
give a hint about what I ḿ doing wrong?
rsync command )some date-time variable for files, but that is working):
rsync -axhhX --no-perms --omit-dir-times\
--backup --backup-dir=$NiceDate/updated --suffix=.$Now\
--delete --delete-excluded\
--backup-dir-dels=$NiceDate/deleted --suffix-dels=.`date +%H%M`\
--log-file=/home/hans/.mybackup/backup.log.$RawDate-$Now\
--include-from=/home/hans/.mybackup/include.txt\
--exclude-from=/home/hans/.mybackup/exclude.txt\
/home/hans /mnt/backup
===
2nd, similar question:
to play around with this stuff I wanted to test with the hidden files only
(.*), so I tried to add all non-hidden directories and normal files to this
exclude-test.txt, but did not get it working (it worked too much):
exclude-test.txt
.*
.mybackup/backup.log.*
[a-z]*
[A-Z]*
while my include-test is:
.kde4/share/apps/kabc/distlists
.kde4/share/apps/kabc/std.vcf
.mozilla/
.mozilla/firefox/
.mozilla/firefox/plp6wzjm.default/
.mozilla/firefox/plp6wzjm.default/bookmarkbackups/*
.mybackup
In this case I don't get any files backed up (dry-run, so checked in log-file
only).
Hmm, you should get
.mozilla/firefox/plp6wzjm.default/bookmarkbackups/*
.mybackup/
backed up. Otherwise the rest will be hidden from transfer (to be
precise, any file/directory starting with a letter or .).
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
