Re: Include Exclude .. a canonical way
Harry Putnam <[EMAIL PROTECTED]> writes: > Here are some of what hasn't worked in an EXCLUDE file. > > + /.kde3.5/share/apps/konqueror/bookmarks.xml > /.kde3.5/ > > + .kde3.5/share/apps/konqueror/bookmarks.xml > /.kde3.5** > > + .kde3.5/share/apps/konqueror/bookmarks.xml > .kde3.5/ > > There are a number of other combos I've tried but still not getting > the results I want. > > The coverage in `man rsync' seems too general to get a working example > of doing this. That last bit about the manual being too general was wrong. It does include an example that let me find the solution but first one more failure should be listed. This fails: + /.kde3.5/share/ + /.kde3.5/share/apps/ + /.kde3.5/share/apps/konqueror/ + /.kde3.5/share/apps/konqueror/bookmarks.xml - /.kde3.5/* However using a double asterisk makes it all work: + /.kde3.5/share/ + /.kde3.5/share/apps/ + /.kde3.5/share/apps/konqueror/ + /.kde3.5/share/apps/konqueror/bookmarks.xml - /.kde3.5/* Moves only the bookmarks file Still it seems there would be a more succinct way... -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Wed, Feb 22, 2006 at 05:14:29AM -0600, Harry Putnam wrote: > Still it seems there would be a more succinct way... There is if you use rsync 2.6.7 (currently in release testing): rsync -avO --prune-empty-dirs --include=bookmarks.xml --filter='-! */' ~/.kde* some:dest/ This tells rsync to descend through all the ~/.kde* directories that exist on the sender side and look for one or more bookmarks.xml files. The --prune-empty-dirs option is new, and ensures that you only create the directories necessary to hold the bookmarks.xml files. Notes: the filter option means "exclude everything that is not a directory", and the -O option avoids changing the modify-time on the directories. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
Wayne Davison <[EMAIL PROTECTED]> writes: > On Wed, Feb 22, 2006 at 05:14:29AM -0600, Harry Putnam wrote: >> Still it seems there would be a more succinct way... > > There is if you use rsync 2.6.7 (currently in release testing): > > rsync -avO --prune-empty-dirs --include=bookmarks.xml > --filter='-! */' ~/.kde* some:dest/ > > This tells rsync to descend through all the ~/.kde* directories that > exist on the sender side and look for one or more bookmarks.xml files. > The --prune-empty-dirs option is new, and ensures that you only create > the directories necessary to hold the bookmarks.xml files. Notes: the > filter option means "exclude everything that is not a directory", and > the -O option avoids changing the modify-time on the directories. Nice .. since I am testing 6.7. I just built it yesterday and installed in a test --prefix.. The code I posted did already create only those dirs necessary to house bookmarks.xml but it took a lot more dinking around to find the right directories to include and so forth. This looks less cumbersome and time consuming. It will require close attention to the man page and experimentation though. And I'll have to move the test install into production. For me production is not too serious though... just a home boy single user with lots of data. That seems to be the big problem for me. I don't look at rsync for long periods and then when the time comes to finally get into it, I've forgotten all I ever picked up and end up back here flumoxed by the ins and outs of include/exclude. I think I'm probably being too picky about what gets backedup and should probably just accept the room use hit and and get disked up until I'm ahead of the 8 ball. If one just backs up by directory ignoring what may be in it, all of a sudden the rules get real easy.. that is, there are none -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Wed, 2006-02-22 at 11:24 -0800, Wayne Davison wrote: > rsync -avO --prune-empty-dirs --include=bookmarks.xml > --filter='-! */' ~/.kde* some:dest/ I just discovered a way to transfer certain files and folders a few levels below a generic exclude rule without having to include everything on the way; in hindsight it seems so obvious that I'm really surprised no one has discovered it yet. End your exclude rules in * instead of **, use --relative, and just list all the things you want included! The source arguments "sneak under" the generic exclude rule. Example: rsync --exclude=/.kde* --relative ~/./ \ ~/./.kde3.5/share/apps/konqueror/bookmarks.xml dest/ If you need to transfer a file or folder that matches a generic exclude rule instead of merely being under a generic exclude rule, you need a single higher-priority include rule specific to that file or folder, just as you would if you didn't list the file or folder as a source path. If you just generate a source argument and a single high-priority specific include rule for each thing you want to transfer, rsync will do what you expect. I've tried this approach with rsync-acl 2.6.7pre2 and it seems to work. Note the ./ that marks the start of --relative path information; this syntax was introduced in 2.6.7. This approach improves upon the use of --prune-empty-dirs in that the latter might omit empty directories that you actually want; it also handles 99% of the cases that the "traverse" sender filters I proposed would handle. -- Matt McCutchen [EMAIL PROTECTED] http://hashproduct.metaesthetics.net/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
Matt McCutchen <[EMAIL PROTECTED]> writes: > On Wed, 2006-02-22 at 11:24 -0800, Wayne Davison wrote: >> rsync -avO --prune-empty-dirs --include=bookmarks.xml >>--filter='-! */' ~/.kde* some:dest/ > > I just discovered a way to transfer certain files and folders a few > levels below a generic exclude rule without having to include everything > on the way; in hindsight it seems so obvious that I'm really surprised > no one has discovered it yet. End your exclude rules in * instead of > **, use --relative, and just list all the things you want included! The > source arguments "sneak under" the generic exclude rule. Example: > rsync --exclude=/.kde* --relative ~/./ \ > ~/./.kde3.5/share/apps/konqueror/bookmarks.xml dest/ Nice, but what else might the --relative inclusion do? I remember some time in the past having that set in rsnaphshot config and ending up with a really confusing directory structure under the backup dest. (May have had something like --copy-unsafe-links in there too) Maybe if I leave out any `link' rules (which I currently do) it will have no effect other than the one you describe? -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Sat, 2006-02-25 at 10:27 -0600, Harry Putnam wrote: > Matt McCutchen <[EMAIL PROTECTED]> writes: > > rsync --exclude=/.kde* --relative ~/./ \ > > ~/./.kde3.5/share/apps/konqueror/bookmarks.xml dest/ > > Nice, but what else might the --relative inclusion do? I remember some > time in the past having that set in rsnaphshot config and ending up > with a really confusing directory structure under the backup dest. All --relative does is cause rsync to duplicate the source path (minus any leading /) inside the destination. For example, the command cd / && rsync usr/ home/matt/ /backup/ mixes my personal files with bin, lib, share, src, and so forth immediately inside the backup folder. But cd / && rsync --relative usr/ home/matt/ /backup/ creates /backup/usr and /backup/home/matt . --relative is useful when you are copying multiple sources and want them to wind up in the same locations relative to each other in the destination, whether you copy all the sources in one rsync command or each source in its own command. Rsnapshot passes --relative by default and runs a separate rsync command for each "backup" entry in the configuration file. Given this configuration file: snapshot_root /snapshots/ backup /home/ localhost/ backup /etc/ localhost/ backup /usr/local/ localhost/ rsnapshot runs: rsync --relative /home/ /snapshots/localhost/ rsync --relative /etc/ /snapshots/localhost/ rsync --relative /usr/local/ /snapshots/localhost/ and the following directories are created: /snapshots/localhost/home /snapshots/localhost/etc /snapshots/localhost/usr/local A configuration file like this... snapshot_root /snapshots/ backup /home/ localhost/home/ backup /etc/ localhost/etc/ backup /usr/local/ localhost/usr/local/ will create the following "really confusing directory structure" unless you turn --relative off by setting rsync_long_args: /snapshots/localhost/home/home /snapshots/localhost/etc/etc /snapshots/localhost/usr/local/usr/local So --relative is useful if you set the destination path appropriately. However, in some cases you may only want to duplicate a suffix of the source path inside the destination; to do that, put a ./ in front of that suffix. The command rsync --exclude=/.kde* --relative ~/ \ ~/.kde3.5/share/apps/konqueror/bookmarks.xml /dest/ would create /dest/home/harry/ and /dest/home/harry/.kde3.5/share/ apps/konqueror/bookmarks.xml , but your original command matched your home directory with /dest/ itself. Using ./ gives the desired behavior: rsync --exclude=/.kde* --relative ~/./ \ ~/./.kde3.5/share/apps/konqueror/bookmarks.xml /dest/ Now you get /dest/.kde3.5/share/apps/konqueror/bookmarks.xml because the part of the source path after the ./ is duplicated inside the destination. -- Matt McCutchen [EMAIL PROTECTED] http://hashproduct.metaesthetics.net/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
Matt McCutchen <[EMAIL PROTECTED]> writes: > All --relative does is cause rsync to duplicate the source path (minus > any leading /) inside the destination. For example, the command > cd / && rsync usr/ home/matt/ /backup/ > mixes my personal files with bin, lib, share, src, and so forth > immediately inside the backup folder. But > cd / && rsync --relative usr/ home/matt/ /backup/ > creates /backup/usr and /backup/home/matt . Thanks for the complicated explanation... As you've guessed I have a pretty thin understanding of how that works. I haven't really understood all you had to say yet and will need to go thru it a few times and experiment with it. I always have trouble with visualizing things... probaby partially dyslexic or something. However It can do more than your saying there I think when symlinks are involved it can have unexpected results. See this thread on gmane that nobody responded too. You'll notice I had an even murkier understanding then but still what was happening was a "really confusing directory structure": http://thread.gmane.org/gmane.comp.sysutils.backup.rsnapshot.general/742 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Sat, 2006-02-25 at 19:20 -0600, Harry Putnam wrote: > However It can do more than your saying there I think when symlinks > are involved it can have unexpected results. > > See this thread on gmane that nobody responded too. You'll notice I > had an even murkier understanding then but still what was happening > was a "really confusing directory structure": > > http://thread.gmane.org/gmane.comp.sysutils.backup.rsnapshot.general/742 >From that thread: > The whole SOURCE path is mirrored in there [...]. You have captured the effect of --relative in one sentence. That's all there is to it! As far as I can tell, the symlink from /home/reader to /anex2/reader had nothing to do with the behavior because your rsnapshot.conf never mentioned /home/reader. On the other hand, if /anex2/reader had been a symlink, some strange things might have happened. If you want the details of the interactions between symlinks and --relative, read the new explanation of --no-implied-dirs in the CVS rsync man page: http://www.samba.org/cvsweb/rsync/rsync.yo Playing with rsync is probably the best way to figure out what it does in different cases. In fact, I have a directory ~/tmp dedicated to rsync experimentation; an assortment of source and destination directories has accumulated inside it as I have investigated various behaviors and issues. -- Matt McCutchen [EMAIL PROTECTED] http://hashproduct.metaesthetics.net/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Sat, Feb 25, 2006 at 08:54:46PM -0500, Matt McCutchen wrote: > http://www.samba.org/cvsweb/rsync/rsync.yo If you'd prefer to see the man page in HTML format instead of raw yodl, the version from the last "nightly" tar file is always found here: http://rsync.samba.org/ftp/rsync/nightly/rsync.html ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
