Re: rsync read block size

2009-05-23 Thread Ming Zhang
No. The block size is NEVER relevant for a copy via rsync, bits are bits regardless of you copying them bit by bit or in larger groups. please have a read http://www.oracle.com/technology/documentation/berkeley-db/db/ref/transapp/archival.html see the db page size issue when copy a online db.

Re: rsync read block size

2009-05-23 Thread Ming Zhang
On Sat, May 23, 2009 at 10:25 AM, Matthias Schniedermeyer m...@citd.de wrote: On 23.05.2009 08:44, Ming Zhang wrote: No. The block size is NEVER relevant for a copy via rsync, bits are bits regardless of you copying them bit by bit or in larger groups. please have a read http

rsync read block size

2009-05-22 Thread Ming Zhang
Hi All We want to use rsync to backup a live Berkley db to a remote site. BDB has a requirement that read has to be in the unit of db page size. So wonder how could we make sure that rsync can follow that? If we need to change the code, where we should begin to look at? Thanks! Ming -- Please

Re: rsync read block size

2009-05-22 Thread Ming Zhang
it become a BDD question now.. ;) On Fri, May 22, 2009 at 5:37 PM, Matthias Schniedermeyer m...@citd.de wrote: On 22.05.2009 16:25, Ming Zhang wrote: Hi All We want to use rsync to backup a live Berkley db to a remote site. BDB has a requirement that read has to be in the unit of db page

--files-from standard input

2008-10-20 Thread Ming Zhang
Hi All If rsync gets files from standard input, when it starts to process that file, immediately or read until EOF, or somewhere in between? For example, if a list is generated slowly but continuously and keep feeding to rsync, will rsync wait a very long time before ever handling the first

Re: --files-from standard input

2008-10-20 Thread Ming Zhang
On Mon, 2008-10-20 at 15:12 -0400, Matt McCutchen wrote: On Mon, 2008-10-20 at 14:27 -0400, Ming Zhang wrote: If rsync gets files from standard input, when it starts to process that file, immediately or read until EOF, or somewhere in between? For example, if a list is generated slowly

Re: writefd_unbuffered failed to write 4 bytes

2008-04-29 Thread Ming Zhang
. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881

rsync daemon did not chdir back after chdir in module path

2008-03-25 Thread Ming Zhang
issue or something worthy fix? not sure what else information i need to provide. thanks. -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change

Re: untransferred files

2008-03-14 Thread Ming Zhang
On Fri, 2008-03-14 at 12:27 -0400, Matt McCutchen wrote: On Thu, 2008-03-13 at 11:01 -0400, Ming Zhang wrote: Some time rsync just return error code 23 that some files are not transferred. Is there a way to get a list of these files so we can retry it later time? For example, we scan

untransferred files

2008-03-13 Thread Ming Zhang
use files-from which is much faster. Thanks. -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo

Re: untransferred files

2008-03-13 Thread Ming Zhang
On Thu, 2008-03-13 at 18:27 -0700, Wayne Davison wrote: On Thu, Mar 13, 2008 at 11:01:35AM -0400, Ming Zhang wrote: Some time rsync just return error code 23 that some files are not transferred. Is there a way to get a list of these files so we can retry it later time? The list of files

Re: Windows Trouble with --link-dest set: file not found when rsync tries to create hard link

2008-02-06 Thread Ming Zhang
)454,656 bytes 0 Dir(s) 27,704,442,880 bytes free -- Paul E. Johnson Professor, Political Science 1541 Lilac Lane, Room 504 University of Kansas -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com

Re: Windows Trouble with --link-dest set: file not found when rsync tries to create hard link

2008-02-06 Thread Ming Zhang
On Wed, 2008-02-06 at 13:57 -0600, Paul Johnson wrote: On Feb 6, 2008 9:45 AM, Ming Zhang [EMAIL PROTECTED] wrote: On Wed, 2008-02-06 at 00:14 -0600, Paul Johnson wrote: I have great luck running rsync from one linux system to another, or from a windows system (with cwRsync) to a linux

Re: how to know the detail about a partial transfer?

2008-01-25 Thread Ming Zhang
. And processing / detecting errors by scripts becomes very simple. you are right. that is what i am doing right now. prefix each message with all special prefix, easier to grep. Paul Slootman -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com

how to know the detail about a partial transfer?

2008-01-24 Thread Ming Zhang
Hi All We always run rsync with --files-from and sometime it gets a error code 23, partial transfer. is it possible to find out what files are not transferred without check the error message by human eye(since we want to do it in a script)? Thanks. -- Ming Zhang @#$%^ purging memory

Re: how to know the detail about a partial transfer?

2008-01-24 Thread Ming Zhang
On Thu, 2008-01-24 at 20:20 -0500, Matt McCutchen wrote: On Thu, 2008-01-24 at 18:00 -0500, Ming Zhang wrote: We always run rsync with --files-from and sometime it gets a error code 23, partial transfer. is it possible to find out what files are not transferred without check the error

Re: how to know the detail about a partial transfer?

2008-01-24 Thread Ming Zhang
On Thu, 2008-01-24 at 21:25 -0500, Matt McCutchen wrote: On Thu, 2008-01-24 at 20:56 -0500, Ming Zhang wrote: Not easily, since the error messages indicating different kinds of failure for a given file don't follow a standard pattern. Some approaches: is it possible to change those

Re: rsync 3.00pre6 segfault in add_dirs_to_tree

2008-01-14 Thread Ming Zhang
ps, not sure if you saw the half open socket issue i reported last month? will you fix it or leave it there? Ming On Mon, 2008-01-14 at 17:12 -0500, Matt McCutchen wrote: On Fri, 2007-12-14 at 13:34 -0500, Ming Zhang wrote: sorry but seems it still crashes somehow. global dir_count

Re: rsync 3.00pre6 segfault in add_dirs_to_tree

2008-01-14 Thread Ming Zhang
Hi Matt No news is good news, so far QA and I have never seen crash again. Thanks a lot for the fix! Ming On Mon, 2008-01-14 at 17:12 -0500, Matt McCutchen wrote: On Fri, 2007-12-14 at 13:34 -0500, Ming Zhang wrote: sorry but seems it still crashes somehow. global dir_count is 23

module name check

2008-01-09 Thread Ming Zhang
to use it. $ rsync vpn lab11-101::abc/def @ERROR: Unknown module 'abc' rsync error: error starting client-server protocol (code 5) at main.c(1383) [sender=2.6.9] Ming -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com

Re: module name check

2008-01-09 Thread Ming Zhang
On Wed, 2008-01-09 at 17:51 +0100, Paul Slootman wrote: On Wed 09 Jan 2008, Ming Zhang wrote: I tried to find out in the man page about what is the valid character we can use in module name in rsyncd.conf but failed. Anyone knows? Well, your example of using a slash is obviously wrong

Re: module name check

2008-01-09 Thread Ming Zhang
Hi Wayne On Wed, 2008-01-09 at 11:45 -0800, Wayne Davison wrote: On Wed, Jan 09, 2008 at 11:25:20AM -0500, Ming Zhang wrote: I defines a module with abc/def as name. I've added code to disallow such a module name and documented the module naming rules in the rsyncd.conf manpage. thanks

Re: Moved/Renamed Files

2008-01-04 Thread Ming Zhang
On Fri, 2008-01-04 at 14:12 -0500, Boris Toloknov wrote: Ming Zhang wrote: On Thu, 2008-01-03 at 20:19 -0500, Boris Toloknov wrote: Hi, It seems that rsync transfers files whose names was changed or which were moved to another directory since the previous synchronization. I

Re: Moved/Renamed Files

2008-01-04 Thread Ming Zhang
configurable size. Thanks. Boris -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com

Re: Moved/Renamed Files

2008-01-04 Thread Ming Zhang
On Fri, 2008-01-04 at 15:05 -0500, Boris Toloknov wrote: Ming Zhang wrote: On Fri, 2008-01-04 at 14:12 -0500, Boris Toloknov wrote: Ming Zhang wrote: On Thu, 2008-01-03 at 20:19 -0500, Boris Toloknov wrote: Hi, It seems that rsync transfers

Re: Moved/Renamed Files

2008-01-04 Thread Ming Zhang
On Fri, 2008-01-04 at 16:21 -0500, Boris Toloknov wrote: Ming Zhang wrote: On Fri, 2008-01-04 at 15:05 -0500, Boris Toloknov wrote: Ming Zhang wrote: On Fri, 2008-01-04 at 14:12 -0500, Boris Toloknov wrote: Ming Zhang wrote

Re: what is the right exit code for some vanished files?

2007-12-24 Thread Ming Zhang
On Mon, 2007-12-24 at 01:27 -0500, Matt McCutchen wrote: On Sun, 2007-12-23 at 15:55 -0500, Ming Zhang wrote: if sender read by path again and get ENOENT, then this ENOENT can tell rsync enough info. (though current rsync might say partial transfer though it is a vanished file

Re: what is the right exit code for some vanished files?

2007-12-23 Thread Ming Zhang
On Thu, 2007-12-20 at 10:24 -0500, Matt McCutchen wrote: On Wed, 2007-12-19 at 22:25 -0500, Ming Zhang wrote: so this file-list is the internal file list, not the --files-from content. Yes, the correctness property is based on the internal file list. rsync's goal is to keep src

Re: what is the right exit code for some vanished files?

2007-12-19 Thread Ming Zhang
On Wed, 2007-12-19 at 21:22 -0500, Matt McCutchen wrote: On Sun, 2007-12-16 at 13:47 -0500, Ming Zhang wrote: even code 24 is in order because the nonexistence of a --files-from entry cannot affect the correctness of a run with --delete in the way that a traditional vanishing can

Re: what is the right exit code for some vanished files?

2007-12-16 Thread Ming Zhang
On Fri, 2007-12-14 at 23:27 -0500, Matt McCutchen wrote: On Fri, 2007-12-14 at 18:24 -0500, Ming Zhang wrote: What should be the right exit code for vanished file in this scenario. (1) Use inotify or other mechanism to check changed files and then generate a file list. (2) file get

Re: Rsync 3.0.0pre7 released

2007-12-16 Thread Ming Zhang
this link: http://rsync.samba.org/ftp/rsync/rsync-3.0.0pre7-NEWS ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ming Zhang @#$%^ purging memory... (*!% http

Re: rsync 3.00pre6 segfault in add_dirs_to_tree

2007-12-14 Thread Ming Zhang
fyi, i check the core dump again, you are 100% right, the global dir_count was 3. Ming On Thu, 2007-12-13 at 07:06 -0800, Wayne Davison wrote: On Wed, Dec 12, 2007 at 09:44:47PM -0500, Ming Zhang wrote: from the mode, it looks that both of them are directories, so S_ISDIR() should be 1

rsync dead loop (almost) forever in read_timeout() if remote host down

2007-12-14 Thread Ming Zhang
rsync have an application level heatbeat to detect such failure? -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org

Re: logical volumes alignment

2007-12-14 Thread Ming Zhang
PROTECTED]:/dev/VG/DestinationLogicalVolume rsync does not work on special file like block device i believe. Do you think could it be possible? Thanks Marco -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881

Re: rsync 3.00pre6 segfault in add_dirs_to_tree

2007-12-14 Thread Ming Zhang
-13 at 07:06 -0800, Wayne Davison wrote: On Wed, Dec 12, 2007 at 09:44:47PM -0500, Ming Zhang wrote: from the mode, it looks that both of them are directories, so S_ISDIR() should be 1 and thus dir_cnt-- should get executed, but later show dir_cnt is still 1. I wonder if dir_cnt was 3 when

what is the right exit code for some vanished files?

2007-12-14 Thread Ming Zhang
on some files and then set exit code to 23. but shall rsync return 24? -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https

Re: rsync 3.00pre6 segfault in add_dirs_to_tree

2007-12-13 Thread Ming Zhang
On Thu, 2007-12-13 at 07:06 -0800, Wayne Davison wrote: On Wed, Dec 12, 2007 at 09:44:47PM -0500, Ming Zhang wrote: from the mode, it looks that both of them are directories, so S_ISDIR() should be 1 and thus dir_cnt-- should get executed, but later show dir_cnt is still 1. I wonder

rsync 3.00pre6 segfault in add_dirs_to_tree

2007-12-12 Thread Ming Zhang
binary or other information if need. Thanks! -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo

memory leak in print_rsync_version

2007-12-11 Thread Ming Zhang
Licence for details.\n); + +#if SUBPROTOCOL_VERSION != 0 + free(subprotocol); +#endif } -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change

Re: Detecting reason for rsync errors

2007-11-19 Thread Ming Zhang
/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https

better rsync error code when no space?

2007-11-12 Thread Ming Zhang
=io.c, line=601): about to call exit(12) looks like error code 12 is caused by first error, 104. shall rsync report ENOSPC for this? so it can be much easier to trace and find out the error. thanks! -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http

Re: can rsync scan files only with mtime since T?

2007-08-29 Thread Ming Zhang
On Wed, 2007-08-29 at 16:16 -0400, Matt McCutchen wrote: On 8/24/07, Ming Zhang [EMAIL PROTECTED] wrote: performance wise, does rsync and unison has fundamentally different? because i saw unison document also mention delta detection algorithm like rsync. Unison does use the same delta

Re: can rsync scan files only with mtime since T?

2007-08-24 Thread Ming Zhang
On Fri, 2007-08-24 at 09:47 -0400, Matt McCutchen wrote: On 8/23/07, Ming Zhang [EMAIL PROTECTED] wrote: this one looks interesting! fire a flame here, which one is better, unison or rsync, for daily backup purpose? Fundamentally, rsync does stateless one-way copying and Unison does

can rsync scan files only with mtime since T?

2007-08-23 Thread Ming Zhang
one day ago (so is modified since yesterday backup). This can greatly reduce the WAN traffic. Is this doable with current rsync? Thanks! -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881

Re: can rsync scan files only with mtime since T?

2007-08-23 Thread Ming Zhang
On Thu, 2007-08-23 at 18:25 -0400, Matt McCutchen wrote: On 8/23/07, Ming Zhang [EMAIL PROTECTED] wrote: I have a file system that contains millions of small files. Since I backup it everyday with rsync using slow WAN link, I think it will be nice that if rsync can do this: An option

Re: can rsync scan files only with mtime since T?

2007-08-23 Thread Ming Zhang
:) regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com [EMAIL PROTECTED] -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881

huge number of files

2007-06-22 Thread Ming Zhang
on this. Thanks! -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http

Re: huge number of files

2007-06-22 Thread Ming Zhang
On Fri, 2007-06-22 at 17:04 -0400, Matt McCutchen wrote: On 6/22/07, Ming Zhang [EMAIL PROTECTED] wrote: I wonder if rsync can have an option that when it scan the file system tree and accumulate to N number of files, it process these files before scanning further. The current CVS rsync

Re: huge number of files

2007-06-22 Thread Ming Zhang
On Fri, 2007-06-22 at 17:07 -0400, Ming Zhang wrote: On Fri, 2007-06-22 at 17:04 -0400, Matt McCutchen wrote: On 6/22/07, Ming Zhang [EMAIL PROTECTED] wrote: I wonder if rsync can have an option that when it scan the file system tree and accumulate to N number of files, it process

Re: huge number of files

2007-06-22 Thread Ming Zhang
On Fri, 2007-06-22 at 21:54 +0200, Christoph Biedl wrote: Ming Zhang wrote... I wonder if rsync can have an option that when it scan the file system tree and accumulate to N number of files, it process these files before scanning further. I'd suggest to run rsync in each directory

preserve EA?

2007-05-09 Thread Ming Zhang
Hi all When rsync replicate one file to remote side, will the extended attributes be copied as well? Thanks, Ming -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: preserve EA?

2007-05-09 Thread Ming Zhang
On Wed, 2007-05-09 at 14:51 -0400, Ming Zhang wrote: Hi all When rsync replicate one file to remote side, will the extended attributes be copied as well? Thanks, sorry that i should do more check. the rsync man page on site does not have this while fc6 rsync man page has -X support. i guess

Re: preserve EA?

2007-05-09 Thread Ming Zhang
On Wed, 2007-05-09 at 16:07 -0400, Matt McCutchen wrote: On 5/9/07, Ming Zhang [EMAIL PROTECTED] wrote: sorry that i should do more check. the rsync man page on site does not have this while fc6 rsync man page has -X support. i guess there are some extra patches floating around

Re: preserve EA?

2007-05-09 Thread Ming Zhang
On Wed, 2007-05-09 at 16:32 -0400, Matt McCutchen wrote: On 5/9/07, Ming Zhang [EMAIL PROTECTED] wrote: side question, if i know one file only have EA changed, thus mtime is updated, can i force the rsync to do EA update only? regular rsync run will do checksum stuff if mtime changed