Re: Are there any current, in play plans, to multi-thread rsync?

2009-10-17 Thread Wayne Davison
On Fri, Oct 16, 2009 at 8:32 PM, Michael Brian - IL micha...@tusc.comwrote: I am looking for an enhancement to multi-thread or parallelize rsync. Are there any plans that would allow one to set a parameter, like 'number_of_threads' and rsync will ship multiple files at the same time? Not at

Re: retrieve files without local dir compare

2009-10-17 Thread Wayne Davison
On Wed, Oct 14, 2009 at 2:36 AM, Marc Mertes mer...@uni-bonn.de wrote: If the data is here, it is imported by a special software, after the import it will be deleted from that directory. The deleting can't be disabled. If you know what file just got processed, append its name onto an exclude

RE: Are there any current, in play plans, to multi-thread rsync?

2009-10-17 Thread Michael Brian - IL
Thanks for the update. We are using rsync to sync up the Oracle archived transaction logs to our standby databases. We would like to control the number of files that could be transfered in parallel. Thanks for the update. Brian -- Brian P Michael Technical Management

Re: Disable checksumming to improve local performance?

2009-10-17 Thread Wayne Davison
On Fri, Sep 04, 2009 at 10:26:43AM -0700, Greg Siekas wrote: Is it possible to disable checksumming? I'm using rsync with -W, whole file, so if a file has changed I don't want to just transfer the changes. There is not currently a way to do that. I whipped up the appended patch that makes

Re: Enhanced authentication and authorization in rsyncd

2009-10-17 Thread Wayne Davison
On Sun, Aug 30, 2009 at 12:06:21PM +0300, Amir Rapson wrote: A slightly better patch file (removed some warnings). Thanks! Sorry for the slow reply, but your patch looks very useful. I'm reviewing it for inclusion in 3.1.0. ..wayne.. -- Please use reply-all for most replies to avoid omitting

Re: Problem with 3.0.6 and fileflags

2009-10-17 Thread Wayne Davison
On Wed, Jul 29, 2009 at 03:24:57PM -0400, Roger Bailey wrote: I am having trouble compiling rsync 3.0.6 on Red hat Linux EL 5.3. [...] I am puzzled by the last entry, no file-flags. That file-flags support is for BSD, so configure determines that it isn't available for Linux. Linux has

Re: How could be total transferred file size more than total file size ?

2009-10-17 Thread Wayne Davison
On Wed, Jul 29, 2009 at 11:17:34AM +0530, paresh masani wrote: Could you please tell me how would it be possible that total transferred file size is more than total file size ? I believe that the only way this can happen is for a resend to occur (when rsync notices that a file doesn't match the

Re: Nice little performance improvement

2009-10-17 Thread Mike Connell
Hi, Interesting. If you're not using incremental recursion (the default in rsync = 3.0.0), I can see that the du would help by forcing the destination I/O to overlap the file-list building in time. But with incremental recursion, the du shouldn't be necessary because rsync actually overlaps

Re: Nice little performance improvement

2009-10-17 Thread Darryl Dixon - Winterhouse Consulting
Hi, In order to expeditiously move these new files offsite, we use a modified version of pyinotify to log all added/altered files across the entire filesystem(s) and then every five minutes feed the list to rsync with the --files-from option. This works very effectively and quickly.

Re: Nice little performance improvement

2009-10-17 Thread Mike Connell
No, not if the file cache isn't large enough for the number of files. E.g. if you have 20 million files and only 256MB RAM, it's likely a bad idea. Splitting down to the subsub (2-levels down) directory level allows a single subsub rsync to fit for me. Warming the cache is beneficial here, I

Re: Nice little performance improvement

2009-10-17 Thread Jamie Lokier
Mike Connell wrote: Hi, Interesting. If you're not using incremental recursion (the default in rsync = 3.0.0), I can see that the du would help by forcing the destination I/O to overlap the file-list building in time. But with incremental recursion, the du shouldn't be necessary because

[SCM] The rsync repository. branch, master, updated. v3.0.3-237-g20caffd

2009-10-17 Thread Rsync CVS commit messages
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project The rsync repository.. The branch, master has been updated via 20caffd2b361bcad51692998411e4cc566c04b40 (commit) from

[SCM] The rsync repository. branch, master, updated. v3.0.3-239-g6f098b0

2009-10-17 Thread Rsync CVS commit messages
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project The rsync repository.. The branch, master has been updated via 6f098b0f8c1c3582013f20970bf575ab487f6bda (commit) via

[SCM] The rsync repository. branch, master, updated. v3.0.3-240-gd23cc15

2009-10-17 Thread Rsync CVS commit messages
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project The rsync repository.. The branch, master has been updated via d23cc156aa36135a2970321873798d35626d477b (commit) from

[SCM] The rsync repository. branch, master, updated. v3.0.3-241-g0a9fbe1

2009-10-17 Thread Rsync CVS commit messages
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project The rsync repository.. The branch, master has been updated via 0a9fbe17de7d9d298ed64263a4b3cfb77b871199 (commit) from