Re: [Bug 11378] New: Please add a '--line-buffered' option to rsync to make logging/output more friendly with pipes/syslog/CI systems/etc.

2015-07-02 Thread Ken Chase
How about <@andrewTO> alias unbuf='stdbuf -i0 -o0 -e0' then unbuf rsync i have not tested this in any way. --progress would be some interesting stuff to parse, esp with all the screen redrawing of the K/s line as well as background deletes and scans overwriting while --progress of the previous

[Bug 11378] New: Please add a '--line-buffered' option to rsync to make logging/output more friendly with pipes/syslog/CI systems/etc.

2015-07-02 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11378 Bug ID: 11378 Summary: Please add a '--line-buffered' option to rsync to make logging/output more friendly with pipes/syslog/CI systems/etc. Product: rsync Version:

Re: cut-off time for rsync ?

2015-07-02 Thread Ken Chase
On Wed, Jul 01, 2015 at 02:05:50PM +0100, Simon Hobson said: >As I read this, the default is to look at the file size/timestamp and if they match then do nothing as they are assumed to be identical. So unless you have specified this, then files which have already been copied should be igno

Re: cut-off time for rsync ?

2015-07-02 Thread Ken Chase
Yes if rsync could keep a 'last state file' that'd be great, which would require the target be unchanged by any other process/usage - this is however the case with many of our uses here - as a backup only target. Then it could just load the target statefile, and only scan the source for changes vs

Re: cut-off time for rsync ?

2015-07-02 Thread Dirk van Deun
> What is taking time, scanning inodes on the destination, or recopying the > entire > backup because of either source read speed, target write speed or a slow > interconnect > between them? It takes hours to traverse all these directories with loads of small files on the backup server. That is

Re: cut-off time for rsync ?

2015-07-02 Thread Mark
You could use find to build a filter to use with rsync, then update the filter every few days if it takes too long to create. I have used a script to build a filter on the source server to exclude anything over 5 days old, invoked when the sync starts, but it only parses around 2000 files per