Re: exclude all but specifice files in dir
Matt McCutchen [EMAIL PROTECTED] writes: On Wed, 2006-02-15 at 18:33 -0600, Harry Putnam wrote: Wayne Davison [EMAIL PROTECTED] writes: On Tue, Feb 14, 2006 at 03:09:48AM -0600, Harry Putnam wrote: - /.kde* That doesn't exclude anything, since you already included things that match that. You should be using - * to exclude everything else other than what was explicitly included. Can I do something that is isolated to ~/.kde? How about - /.kde** ? That will exclude anything under .kde that is not matched by an earlier include rule. I guess that was a good try but it doesn't work for me. These entries in rsync_exclude: + .kde*/share/apps/konqueror/bookmarks.xml /.kde** When the source is ~/ Results in NOTHING under .kde* being moved to dest -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Include Exclude .. a canonical way
I'm wrestling with include/exclude rules and not finding a way to do this: Simplified command: rsync -avv --exclcude-from=rsync_exclude \ ~/ /dest/ Backup ~/ including ~/.kde3.5/share/apps/konqueror/bookmarks.xml but excluding everything else under ~/.kde3.5 Here are some of what hasn't worked in an EXCLUDE file. + /.kde3.5/share/apps/konqueror/bookmarks.xml /.kde3.5/ + .kde3.5/share/apps/konqueror/bookmarks.xml /.kde3.5** + .kde3.5/share/apps/konqueror/bookmarks.xml .kde3.5/ There are a number of other combos I've tried but still not getting the results I want. The coverage in `man rsync' seems too general to get a working example of doing this. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
Harry Putnam [EMAIL PROTECTED] writes: Here are some of what hasn't worked in an EXCLUDE file. + /.kde3.5/share/apps/konqueror/bookmarks.xml /.kde3.5/ + .kde3.5/share/apps/konqueror/bookmarks.xml /.kde3.5** + .kde3.5/share/apps/konqueror/bookmarks.xml .kde3.5/ There are a number of other combos I've tried but still not getting the results I want. The coverage in `man rsync' seems too general to get a working example of doing this. That last bit about the manual being too general was wrong. It does include an example that let me find the solution but first one more failure should be listed. This fails: + /.kde3.5/share/ + /.kde3.5/share/apps/ + /.kde3.5/share/apps/konqueror/ + /.kde3.5/share/apps/konqueror/bookmarks.xml - /.kde3.5/* However using a double asterisk makes it all work: + /.kde3.5/share/ + /.kde3.5/share/apps/ + /.kde3.5/share/apps/konqueror/ + /.kde3.5/share/apps/konqueror/bookmarks.xml - /.kde3.5/* Moves only the bookmarks file Still it seems there would be a more succinct way... -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsyncd log file permission
Hi I am wondering if there is any way to change the file permission of rsync.log and rsyncd.lock via rsyncd.conf. Most likely the lock file is not possible, but maybe the other file is possible. The reason I am asking is because if I UserA runs rsync, then UserB won't be able to write to rsync.log file since its owned by UserA and readable only to UserB. So I'm trying to make it writable by all when performing a sync. Is it possible to have log file = /logs/cdersync/$USER.rsync.log lock file = /logs/$USER.rsyncd.lock ?? From current rsyncd.conf log file = /logs/cdersync/rsync.log lock file = /logs/rsyncd.lock -rw-r--r--1 UserA GroupA10399049 Feb 16 17:55 rsync.log -rw---1 UserA GroupA 0 Dec 20 10:44 rsyncd.lock -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
propagate atimes with rsync-2.6.6
Dear Wayne. Thank you very much for your extensive comments on the our patch. I will try and take them into account and mend our patch accordingly. Unfortunately, I can only do this during my leisure time which is already very limited so it might take some time. But it would be very nice if this feature could make it into the reference version. We consider it as very useful if not necessary for back-ups of systems where the access to files by employees is monitored. That's especially relevant if the usage of a database should be assessed. We also thought of extending the changes such that rsync would also transfer atimes if the modification times are the same but we dropped that for the moment since the hardlink-feature allows this already. I have received an email from Emile LeBlanc who supports the development of these features very much. The initiative for our changes came from a company with daily back-ups of their database. Maybe, this feature may be useful to other people too. Maybe, we could merge the two patches to create a nice new feature. That would be great. Thank you very much, Christian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync help needed...
Hello, I was reading your posts about RSYNC. We have a massive Oracle schema lots of datafiles about 750 GB size. We do rsync datafiles from source to target server but everytime we cleanup the datafiles on the target server and do rsync every 2 weeks. On the target side mostly the datafiles will be same but on source we might have added few datafiles or made some changes in data and as such the size of schema will increase every time since we add new datafiles. If we leave the old datafiles at the target end and rsync will it be faster ? What rysnc command we need to use without cleaning the old files so in the old files only the changes will be copied and new files also needs to be rsynced. What is the rsync command to be used ? Thanks, lsk. -- View this message in context: http://www.nabble.com/Rsync-help-needed...-t1170765.html#a3075671 Sent from the Samba - rsync forum at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: exclude all but specifice files in dir
On Wed, Feb 22, 2006 at 03:18:11AM -0600, Harry Putnam wrote: + .kde*/share/apps/konqueror/bookmarks.xml /.kde** You need to restore all the directory includes from your first email: + .kde*/ + .kde*/share/ + .kde*/share/apps/ + .kde*/share/apps/konqueror/ + .kde*/share/apps/konqueror/bookmarks.xml - /.kde** ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync help needed...
Hi Isk, Please see embedded. Gian lsk wrote: Hello, I was reading your posts about RSYNC. We have a massive Oracle schema lots of datafiles about 750 GB size. We do rsync datafiles from source to target server but everytime we cleanup the datafiles on the target server and do rsync every 2 weeks. What do you mean by cleanup the datafiles on the target server? Are you editing files on the target server? On the target side mostly the datafiles will be same but on source we might have added few datafiles or made some changes in data and as such the size of schema will increase every time since we add new datafiles. If we leave the old datafiles at the target end and rsync will it be faster ? If the target is not 100% different from the source, yes, keep it! Rsync will upload the differences of the two. If 99% of the file is the same, you just saved 99% of your transfer time. :) See timestamps and checksum in the manual to see what option you need to use in your schema. Checksum may take some time with 3/4 TB of data :-/ What rysnc command we need to use without cleaning the old files so in the old files only the changes will be copied and new files also needs to be rsynced. So you want to update files and transfer new files? How about: rsync -e ssh -avz /source/data/files [EMAIL PROTECTED]:/data/files This command works over ssh (for encryption), uses an archive mode (see manual for details), spits out information verbosely, and compresses files during transfer to speed it up. If you would like to watch it happen, add --progress for cool status view. What is the rsync command to be used ? Thanks, lsk. -- View this message in context: http://www.nabble.com/Rsync-help-needed...-t1170765.html#a3075671 Sent from the Samba - rsync forum at Nabble.com. Is this what you were looking for? Gian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
On Wed, Feb 22, 2006 at 05:14:29AM -0600, Harry Putnam wrote: Still it seems there would be a more succinct way... There is if you use rsync 2.6.7 (currently in release testing): rsync -avO --prune-empty-dirs --include=bookmarks.xml --filter='-! */' ~/.kde* some:dest/ This tells rsync to descend through all the ~/.kde* directories that exist on the sender side and look for one or more bookmarks.xml files. The --prune-empty-dirs option is new, and ensures that you only create the directories necessary to hold the bookmarks.xml files. Notes: the filter option means exclude everything that is not a directory, and the -O option avoids changing the modify-time on the directories. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync help needed...
Gian, What do you mean by cleanup the datafiles on the target server? Are you editing files on the target server? lsk: That means I delete/rm the files from the target and then start rsync. If the target is not 100% different from the source, yes, keep it! Rsync will upload the differences of the two. If 99% of the file is the same, you just saved 99% of your transfer time. :) See timestamps and checksum in the manual to see what option you need to use in your schema. Checksum may take some time with 3/4 TB of data :-/ lsk: This is oracle database the header information(timestamp..etc) on each datafile constantly changes which might be very small change but the data inside most of the datafiles are same they wont change much. New oracle datafiles will be added on the source which needs to be transferred. So what we do is every 2 weeks to refresh target server we remove all datafiles on target and rsync all datafiles again from source. My question is if we leave the older datafiles and rsync will it be faster ? Currently I use rsync -czv c for checksum. Thanks, lsk. -- View this message in context: http://www.nabble.com/Rsync-help-needed...-t1170765.html#a3077044 Sent from the Samba - rsync forum at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 3542] New: rsync --list-only a b/ tries to create b
https://bugzilla.samba.org/show_bug.cgi?id=3542 Summary: rsync --list-only a b/ tries to create b Product: rsync Version: 2.6.7 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P3 Component: core AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] QAContact: [EMAIL PROTECTED] Rsync's --list-only option requires a destination, which is strange because the destination isn't used for anything. Nevertheless, in get_local_name, rsync ensures that the destination is usable, which may involve trying to create the destination as a directory. Luckily, if rsync tries to create the directory, the attempt is caught in do_mkdir and an error message results, but rsync really shouldn't care about the destination at all. The patch I will soon attach causes get_local_name to return NULL immediately if --list-only is given. However, there has to be a better way to ensure the destination is not touched in this situation. Separately, it would be an improvement if the destination argument could be omitted when --list-only is given. (Until I read the man page, I kept trying to run rsync --list-only source/ and wondered why I got a usage error.) -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 3542] rsync --list-only a b/ tries to create b
https://bugzilla.samba.org/show_bug.cgi?id=3542 --- Comment #1 from [EMAIL PROTECTED] 2006-02-22 17:31 MST --- Created an attachment (id=1753) -- (https://bugzilla.samba.org/attachment.cgi?id=1753action=view) Makes get_local_name return NULL if --list-only is given -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync help needed...
On Wed, 2006-02-22 at 11:43 -0800, lsk wrote: lsk: This is oracle database the header information(timestamp..etc) on each datafile constantly changes which might be very small change but the data inside most of the datafiles are same they wont change much. New oracle datafiles will be added on the source which needs to be transferred. So what we do is every 2 weeks to refresh target server we remove all datafiles on target and rsync all datafiles again from source. My question is if we leave the older datafiles and rsync will it be faster ? Yes, very much faster! Rsync has a nice incremental transfer algorithm, which is described in tech_report.tex in the source distribution. The sending rsync will notice that large regions of the target file match large regions of the source file by comparing hash codes and will only send the parts of the source file that are new or different. Rsync will notice that a region of one file is identical to a region of the other even if those regions occur in different places in the files. However, if Oracle does some kind of re-indexing or garbage collection that modifies every Nth byte of the on-disk data file, rsync will transfer the whole file from scratch. Currently I use rsync -czv c for checksum. If each data file's first few bytes (header information) change between rsync transfers, then --checksum buys you nothing. Normally rsync will skip transferring a file if the receiver has a corresponding file of the same size and modtime; --checksum makes both sides read their files entirely and skip transfers based on the MD4 of the file's data. However, since the header information changes, the checksums will never match and rsync will transfer the files, but the transfers will be fast because so little of the files changed. On the other hand, if the modtimes of the files are changing but their data is not, using --checksum will result in a relatively small reduction in network traffic: rsync will notice the files match immediately instead of after comparing a few kilobytes worth of block hashes. However, rsync will read each changed file once to compute the checksum and again to transfer it, so the disk hit increases. Someone correct me if I am mistaken about how incremental transfer and --checksum work. -- Matt McCutchen [EMAIL PROTECTED] http://hashproduct.metaesthetics.net/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Include Exclude .. a canonical way
Wayne Davison [EMAIL PROTECTED] writes: On Wed, Feb 22, 2006 at 05:14:29AM -0600, Harry Putnam wrote: Still it seems there would be a more succinct way... There is if you use rsync 2.6.7 (currently in release testing): rsync -avO --prune-empty-dirs --include=bookmarks.xml --filter='-! */' ~/.kde* some:dest/ This tells rsync to descend through all the ~/.kde* directories that exist on the sender side and look for one or more bookmarks.xml files. The --prune-empty-dirs option is new, and ensures that you only create the directories necessary to hold the bookmarks.xml files. Notes: the filter option means exclude everything that is not a directory, and the -O option avoids changing the modify-time on the directories. Nice .. since I am testing 6.7. I just built it yesterday and installed in a test --prefix.. The code I posted did already create only those dirs necessary to house bookmarks.xml but it took a lot more dinking around to find the right directories to include and so forth. This looks less cumbersome and time consuming. It will require close attention to the man page and experimentation though. And I'll have to move the test install into production. For me production is not too serious though... just a home boy single user with lots of data. That seems to be the big problem for me. I don't look at rsync for long periods and then when the time comes to finally get into it, I've forgotten all I ever picked up and end up back here flumoxed by the ins and outs of include/exclude. I think I'm probably being too picky about what gets backedup and should probably just accept the room use hit and and get disked up until I'm ahead of the 8 ball. If one just backs up by directory ignoring what may be in it, all of a sudden the rules get real easy.. that is, there are none -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 3543] New: [ACL] rsync calls default_perms_for_dir on omitted implied dirs before ensuring they exist
https://bugzilla.samba.org/show_bug.cgi?id=3543 Summary: [ACL] rsync calls default_perms_for_dir on omitted implied dirs before ensuring they exist Product: rsync Version: 2.6.7 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P3 Component: core AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] QAContact: [EMAIL PROTECTED] Run the following in an empty directory: mkdir a touch a/b rsync --relative --no-implied-dirs a/b dest/ The following error messages result (my indentation): default_perms_for_dir: sys_acl_get_file(a, SMB_ACL_TYPE_DEFAULT): No such file or directory, falling back on umask default_perms_for_dir: sys_acl_get_file(a, SMB_ACL_TYPE_DEFAULT): No such file or directory, falling back on umask rsync error: some files could not be transferred (code 23) at main.c(882) [sender] The trouble is that rsync is asking for the default permissions of dest/a before dest/a has been created. Moving the calculation of destination default permissions later in recv_generator, after the creation of implied directories, would probably fix the problem, but I'm not familiar enough with the logic of that 520-line function to know where to put it. Separating this calculation from the fuzzy list calculation would also fix what appears to be a logic error. If --fuzzy is given and --perms is not given and the first file to be received into a directory is not a regular file, parent_dirname will be set to the directory but the fuzzy list will not be updated. When rsync does receive a regular file into the directory, it will assume the fuzzy list does not need to be updated and use the wrong list. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 3542] rsync --list-only a b/ tries to create b
https://bugzilla.samba.org/show_bug.cgi?id=3542 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED --- Comment #2 from [EMAIL PROTECTED] 2006-02-22 18:40 MST --- Thanks! This is now fixed in CVS. Also, rsync no longer returns a usage error when used with one local source arg and no destination: this now implies the --list-only option, just like the comparable situation with a remote source arg. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
preventing rsync transfers
I sync some client stations to a central server at the discretion of the users. They run a single-transfer daemon by logging in via their ssh accounts. All is well. Now I also allow a certain user to rsync this accumulated client data (which end up as subdirectories of a single directory on the server) to an external hard drive. This user connects via ssh and runs a remote shell script on the server. (The script is needed to manage rotation of backups and to make sure the drive is mounted/unmounted properly.) All is well. Now I'm thinking: What if these operations overlap? What can I do to prevent this? Or is this even a problem to begin with? -- Peter __ Find your next car at http://autos.yahoo.ca -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/packaging
Date: Wed Feb 22 19:09:51 2006 Author: wayned Update of /data/cvs/rsync/packaging In directory dp.samba.org:/tmp/cvs-serv12497 Modified Files: release-rsync Log Message: - Improved the way we generate the tar file. - The second run of verify-patches is in a better spot now. - Made various improvements to the output. Revisions: release-rsync 1.4 = 1.5 http://www.samba.org/cgi-bin/cvsweb/rsync/packaging/release-rsync?r1=1.4r2=1.5 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Thu Feb 23 00:38:34 2006 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv26459 Modified Files: main.c Log Message: Rsync no longer returns a usage error when used with one local source arg: this now implies the --list-only option, just like one remote source arg has always done. Revisions: main.c 1.295 = 1.296 http://www.samba.org/cgi-bin/cvsweb/rsync/main.c?r1=1.295r2=1.296 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Thu Feb 23 00:40:58 2006 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv30396 Modified Files: rsync.yo Log Message: Document the revised --list-only behavior and the ability to use rsync with one local source arg (i.e. no destination). Revisions: rsync.yo1.357 = 1.358 http://www.samba.org/cgi-bin/cvsweb/rsync/rsync.yo?r1=1.357r2=1.358 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Thu Feb 23 00:43:17 2006 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv1945 Modified Files: NEWS Log Message: Mention the --list-arg change. Revisions: NEWS1.372 = 1.373 http://www.samba.org/cgi-bin/cvsweb/rsync/NEWS?r1=1.372r2=1.373 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
