Re: rsync speedup - how ?

2009-08-07 Thread Daniel.Li
On Fri, 2009-08-07 at 18:44 +0200, devz...@web.de wrote: > > devz...@web.de wrote: > > > so, instead of 500M i would transfer 100GB over the network. > > > that`s no option. > > > > I don't see how you came up with such numbers. > > If files change completely then I don't see why > > you would tra

RE: How to get first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
> (assuming something unixy, of course -- cygwin stuff might work) > > > -Original Message- > > From: rsync-bounces+tony=servacorp@lists.samba.org > > [mailto:rsync-bounces+tony=servacorp@lists.samba.org] On > > Behalf Of Daniel.Li > > Sent:

How to first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
Dear List, I set up a NetBackup destination on my server. And I backup test1 to NetBackup/test1, test2 to NetBackup/test2. Now I wanna know how many folders that have been backuped. It should be 2 folders, with the name of test1 and test2. But I don't know if there is any way to do so? Any ad

How to get first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
Dear List, I set up a NetBackup destination on my server. And I backup test1 to NetBackup/test1, test2 to NetBackup/test2. Now I wanna know how many folders that have been backuped. It should be 2 folders, with the name of test1 and test2. But I don't know if there is any way to do so? Any ad

-acv option, what will be the default "delete option"?

2009-06-16 Thread Daniel.Li
Dear List, If I use -acv for rsync options, what will I choose as default "delete option"? --deletedelete extraneous files from destination dirs --delete-before receiver deletes before transfer, not during --delete-during receiver deletes during transfer (default)

Re: Key for high CPU usage

2009-06-11 Thread Daniel.Li
Hi Leen, Thanks for your reply. On Thu, 2009-06-11 at 08:24 +0200, Leen Besselink wrote: > Leen Besselink wrote: > > Daniel.Li wrote: > >> Dear List, > >> > >> I'm trying to take a closer look at rsync code, and found when we run > >>

Key for high CPU usage

2009-06-10 Thread Daniel.Li
Dear List, I'm trying to take a closer look at rsync code, and found when we run daemon, it will take a lot of CPU (400Mhz). So I'm interested in Which part of rsync code on ver 3.0.5 consuming CPU a lot? Can anyone here help to lighten me up? So I can try to improve the performance or low the CP

Re: Weird behavior in receive_data function

2009-06-10 Thread Daniel.Li
On Thu, 2009-06-11 at 07:11 +0800, Daniel.Li wrote: > Dear List, > > I'm trying to get diff/removed data and it's offset out. So I write a > functions in receive_data. When I run backup, I found there is a weird > behavior which I don't understand. > > i = rec

Weird behavior in receive_data function

2009-06-10 Thread Daniel.Li
Dear List, I'm trying to get diff/removed data and it's offset out. So I write a functions in receive_data. When I run backup, I found there is a weird behavior which I don't understand. i = recv_token(f_in, &data) will receive (i = -1, offset2 = 0) some where in the middle of the transfer proced

Re: rsync --daemon. Can I open more than one instances?

2009-06-08 Thread Daniel.Li
On Mon, 2009-06-08 at 22:39 -0500, Rahul Nabar wrote: > > If you're not using different partitions on different disks, then > doing > > anything in parallel is probably going to slow it down > (reading/writing > > all over the disk, means more seektime, which means waiting). > > No, its just a sin

Re: what kind attrs are going to be transfered by rsync?

2009-06-04 Thread Daniel.Li
2009-06-04 at 14:44 +0800, Daniel.Li wrote: > Dear list, > > I have met this error. I wanna know what attrs are transfer by rsync > program? > > sent 5045 bytes received 262853 bytes 107159.20 bytes/sec > total size is 130560 speedup is 0.49 > rsync error: some files/at

what kind attrs are going to be transfered by rsync?

2009-06-03 Thread Daniel.Li
Dear list, I have met this error. I wanna know what attrs are transfer by rsync program? sent 5045 bytes received 262853 bytes 107159.20 bytes/sec total size is 130560 speedup is 0.49 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1535) [generato

Re: rsync --daemon. Can I open more than one instances?

2009-06-03 Thread Daniel.Li
On Wed, 2009-06-03 at 20:57 -0500, Rahul Nabar wrote: > Is there a way to speed up rsync by opening more than one daemon in > parallel. daemon will create a process when a connection is established. So the limit is hardware. > I use rsync --daemon to start rsync. I was wondering if > opening m

Re: Which function in generator trigger the file transfer?

2009-06-02 Thread Daniel.Li
code. Thanks in advance. On Tue, 2009-06-02 at 15:31 +0800, Daniel.Li wrote: > Dear List, > > I checked the function "generate_files", which might be simply stand for > generator, but didn't find where triggers the file transfer. As receiver > will use "re

Which function in generator trigger the file transfer?

2009-06-02 Thread Daniel.Li
Dear List, I checked the function "generate_files", which might be simply stand for generator, but didn't find where triggers the file transfer. As receiver will use "read_ndx_and_attrs" to read iflags to judge if it's need to be transfered? If I'm wrong, please correct me. Can anyone help to po

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
On Sun, 2009-05-31 at 23:11 -0400, Matt McCutchen wrote: > On Mon, 2009-06-01 at 07:10 +0800, Daniel.Li wrote: > > On Sun, 2009-05-31 at 14:34 -0400, Matt McCutchen wrote: > > > > And one more thing here: > > > > If you are going to prepare this batch file, it s

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
On Sun, 2009-05-31 at 14:34 -0400, Matt McCutchen wrote: > > And one more thing here: > > If you are going to prepare this batch file, it seems there will be > double the workload of network, see below statements? Is that right? > > > > > rsync -av --only-write-batch=/batches/$DATE > bhost:/backup

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
Dear Wayne, Excellent, that's really what I have expected! Is it stable now? Cause I have found that this feature seems to be unstable before ver 3.0.6. - Fixed a --read-batch hang when rsync is reading a batch file that was created from an incremental-recursion transfer. http://samba.anu

Re: how to output verbose information to console when run rsync --daemon?

2009-05-25 Thread Daniel.Li
I used "--log-file" option, but I just found those "connectbla bla..." info. No other valuable info like rsync client with -v options. Can anyone help here ? On Wed, 2009-05-06 at 11:42 +0800, Daniel.Li wrote: > Dear List, > > option "-v" ca

implement diff backup schema on rsync GPL code.

2009-05-25 Thread Daniel.Li
Dear All, I'm planning to implement diff backup feature on rsync. Currently, I have seen delta data and data from server side. And I have found that there are many case in current code, it seems a lot of work to do to implement this feature. Do you have any ideas, or suggestions on how to simply

Re: Does rsync detect file corruption? -- hard link

2009-05-24 Thread Daniel.Li
On Sun, 2009-05-24 at 22:13 -0400, Matt McCutchen wrote: > On Mon, 2009-05-25 at 10:09 +0800, Daniel.Li wrote: > > On Mon, 2009-05-25 at 09:58 +0800, Daniel.Li wrote: > > > What if a video editor? > > > > > > Lots of work with video files, which is very large

Re: Does rsync detect file corruption? -- hard link

2009-05-24 Thread Daniel.Li
On Mon, 2009-05-25 at 09:58 +0800, Daniel.Li wrote: > On Sun, 2009-05-24 at 16:04 +0200, Mac User FR wrote: > > Hard-linking an > > unchanged dir takes very few place. > > What if a video editor? > > Lots of work with video files, which is very large, about 500MB per

Re: Does rsync detect file corruption? -- hard link

2009-05-24 Thread Daniel.Li
On Sun, 2009-05-24 at 16:04 +0200, Mac User FR wrote: > Hard-linking an > unchanged dir takes very few place. What if a video editor? Lots of work with video files, which is very large, about 500MB per file. Editor only delete or rearrange frames in that file. And then it will be back up 500MB

Re: weird access function in rsync code.

2009-05-24 Thread Daniel.Li
On Fri, 2009-05-22 at 22:06 +0800, Daniel.Li wrote: > On Fri, 2009-05-22 at 13:10 +0200, Paul Slootman wrote: > > On Fri 22 May 2009, Daniel.Li wrote: > > > On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: > > > > > > > > > > access(&quo

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
On Fri, 2009-05-22 at 13:10 +0200, Paul Slootman wrote: > On Fri 22 May 2009, Daniel.Li wrote: > > On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: > > > > > > > > access("/home/admin/test",F_OK) > > > > > > > > Resul

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: > On Fri 22 May 2009, Daniel.Li wrote: > > > When I use "access" function to find out if the folder exist in rsync > > code. I found that it's will NOT act as normal function. > > > > I hav

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
On Fri, 2009-05-22 at 16:14 +0800, Daniel.Li wrote: > Dear List, > When I use "access" function to find out if the folder exist in rsync > code. I found that it's will NOT act as normal function. > > I have tried 2 test: > > a) embedded in rsync function "

weird access function in rsync code.

2009-05-22 Thread Daniel.Li
Dear List, When I use "access" function to find out if the folder exist in rsync code. I found that it's will NOT act as normal function. I have tried 2 test: a) embedded in rsync function "int recv_files(int f_in, char *local_name)" and hard coded with following line. access("/home/admin/test

Re: 2.6.9 and 3.0.5 sync error

2009-05-20 Thread Daniel.Li
root privilege on the both side :( > > Le 20 mai 09 à 13:01, Daniel.Li a écrit : > > > Dear List, > > > > I'm using rsync to sync my data, see below command. And I met this > > error? > > > > I think it might be due to 2.6.9 and 3.0.5 compatible issue.

2.6.9 and 3.0.5 sync error

2009-05-20 Thread Daniel.Li
Dear List, I'm using rsync to sync my data, see below command. And I met this error? I think it might be due to 2.6.9 and 3.0.5 compatible issue. So how to solve the problem? Thanks. rsync -vrtopg --progress --delete --password-file=rsync-password /path/on/pc r...@192.168.0.202::destination bui

Re: Delta backup program planned (problem met for adding files)

2009-05-10 Thread Daniel.Li
On Mon, 2009-05-11 at 10:05 +0800, Daniel.Li wrote: > > When I add an function, compiler prints some error, see below: > > And I have found that proto.h is generated automatically. So what I have > missed here in configuration (adding those six extra files) OK, I have found the

Delta backup program planned (problem met for adding files)

2009-05-10 Thread Daniel.Li
Dear List, Currently, I'm trying to implement a new feature to rsync. Delta back: Well, it simply just backups diff blocks, and the program is able to restore to any backup points. Current problem met for adding files! But I met a newbie problem here. I'm going to add six files, which are dif

Re: Prepared meta directory for one to many sync

2009-05-06 Thread Daniel.Li
On Wed, 2009-05-06 at 09:20 +0200, Lukas Macura wrote: > Hi to all, > > sorry, but I am not subscribed to this list. I have a question, because > I did not find any answer for my problem. If you have so, please let me > know! > > We use rsync to synchronise "one-to-many" workstations. We use our

how to output verbose information to console when run rsync --daemon?

2009-05-05 Thread Daniel.Li
Dear List, option "-v" can output information to console. But it seems no effect with "rsync --daemon". So, how can I output verbose information to console when running with command "rsync --daemon" I also tried log file. Seems didn't work. Or is there any other way to read those information, T

Re: [Code study]append_mode definitions

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 06:44 -0700, Wayne Davison wrote: > On Mon, Apr 27, 2009 at 10:33:02AM +0800, Daniel.Li wrote: > > Can anyone help to explain definitions of variable "append_mode"? > > There are two different type of appending (see the man page) depending > on h

Re: [Code study]should we remove if (motd && *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 13:34 +0200, Petr Uzel wrote: > On Mon, Apr 27, 2009 at 07:27:38PM +0800, Daniel.Li wrote: > > On Mon, 2009-04-27 at 19:21 +0800, Daniel.Li wrote: > > > On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: > > > > On Mo

Re: [Code study]should we remove if (motd && *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 19:21 +0800, Daniel.Li wrote: > On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: > > On Mon 27 Apr 2009, Daniel.Li wrote: > > > > > > I'm confused when we will run into "if (motd && *motd)"? > > > > &

Re: [Code study]should we remove if (motd && *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: > On Mon 27 Apr 2009, Daniel.Li wrote: > > > > I'm confused when we will run into "if (motd && *motd)"? > > > > As I have found that Globals is set 0 during initialization, an

[Code study]should we remove if (motd && *motd) section?

2009-04-27 Thread Daniel.Li
Dear List, I'm confused when we will run into "if (motd && *motd)"? As I have found that Globals is set 0 during initialization, and I didn't find anywhere else assign the value. If so, should we remove this section? Or there might be some where I missed? Please correct me, if I'm wrong. Thank

[Code study]append_mode definitions

2009-04-26 Thread Daniel.Li
Dear List, Can anyone help to explain definitions of variable "append_mode"? I have found this variable is changed in many places, and quite confused. option.c #line549~550 > {"append-verify",0, POPT_ARG_VAL,&append_mode, 2, 0, 0 }, > {"no-append",0, POPT_ARG_VAL,&appe

Re: looking for superlifter souce code and related information

2009-04-24 Thread Daniel.Li
On Fri, 2009-04-24 at 07:34 -0700, Wayne Davison wrote: > On Tue, Apr 21, 2009 at 10:39:58AM +0800, Daniel.Li wrote: > > I just googled "superlifter" and found below link, but I can't get any > > source code. > > Sadly, superlifter never achieved lift-off, so t

Re: Any program flow or software spec for Rsync

2009-04-24 Thread Daniel.Li
On Fri, 2009-04-24 at 07:36 -0700, Wayne Davison wrote: > On Tue, Apr 21, 2009 at 02:47:41PM +0800, Daniel.Li wrote: > > I'm trying to take a look at rsync from code-level. > > Is there any software spec or program flow that I can get, which might > > give me a basic un

Re: Any program flow or software spec for Rsync

2009-04-23 Thread Daniel.Li
Is there any protocol standard or specs that I can get? Draft is also fine. Thanks. On Tue, 2009-04-21 at 14:47 +0800, Daniel.Li wrote: > Dear All, > I'm trying to take a look at rsync from code-level. > Is there any software spec or program flow that I can get, which might >

Re: [Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
On Wed, 2009-04-22 at 10:14 +0200, Paul Slootman wrote: > On Wed 22 Apr 2009, Daniel.Li wrote: > > > > Currently, I read this in clientserver.c, line 148. > > > > motd = lp_motd_file(); > > > > I have googled, but didn't the definition of char *lp_

Re: [Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
On Wed, 2009-04-22 at 15:54 +0800, Daniel.Li wrote: > Dear List, > > Currently, I read this in clientserver.c, line 148. > > motd = lp_motd_file(); > > I have googled, but didn't the definition of char *lp_motd_file(void). > Can anyone help to explain the following

[Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
Dear List, Currently, I read this in clientserver.c, line 148. motd = lp_motd_file(); I have googled, but didn't the definition of char *lp_motd_file(void). Can anyone help to explain the following code segment. Many thanks. #line 147 ~ 160, in clientserver.c, version 3.0.5 > if (!am_client) {

Any program flow or software spec for Rsync

2009-04-20 Thread Daniel.Li
Dear All, I'm trying to take a look at rsync from code-level. Is there any software spec or program flow that I can get, which might give me a basic understanding about the code? Thanks in advance! -- Daniel -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubs

Re: looking for superlifter souce code and related information

2009-04-20 Thread Daniel.Li
There is a thread palnning for superlifter http://lists.samba.org/archive/rsync/2002-August/003361.html But I don't know if there is any action later? Anyone knows? Or maybe I have take a look at rZync :) On Tue, 2009-04-21 at 10:39 +0800, Daniel.Li wrote: > Dear List, > >

looking for superlifter souce code and related information

2009-04-20 Thread Daniel.Li
Dear List, I just googled "superlifter" and found below link, but I can't get any source code. http://superlifter.sourceforge.net/ Any help or info on this is mostly appreciated. And sorry for the trouble, if it's NOT allowed to post here. THanks. -- Daniel -- Please use reply-all for most r

Re: How many differs in rsync algorithm from Andrew Tridgell's thesis

2009-04-16 Thread Daniel.Li
Is there anyone can help me on this? On Thu, 2009-04-16 at 10:13 +0800, Daniel.Li wrote: > Dear List, > > I would like to take a look at rsync's algorithm. As newbie on this > algorithm, I just downloaded Andrew Tridgell's original thesis in 1999. > > So my ques

How many differs in rsync algorithm from Andrew Tridgell's thesis

2009-04-15 Thread Daniel.Li
Dear List, I would like to take a look at rsync's algorithm. As newbie on this algorithm, I just downloaded Andrew Tridgell's original thesis in 1999. So my question is "How many differs in rsync algorithm from Andrew Tridgell's thesis". Thanks in advance. -- Daniel -- Please use reply-all

Re: --disable-iconv default configuration

2009-03-16 Thread Daniel.Li
On Mon, 2009-03-16 at 14:42 +0800, Daniel.Li wrote: > On Mon, 2009-03-16 at 14:08 +0800, Daniel.Li wrote: > > Dear List, > > > > I have a quick question about rsync configuration. > > > > If I just simply use ./configure, does "--iconv" su

Re: --disable-iconv default configuration

2009-03-15 Thread Daniel.Li
On Mon, 2009-03-16 at 14:08 +0800, Daniel.Li wrote: > Dear List, > > I have a quick question about rsync configuration. > > If I just simply use ./configure, does "--iconv" supported by server? > > --disable-iconv disable rsync's --iconv option O

--disable-iconv default configuration

2009-03-15 Thread Daniel.Li
Dear List, I have a quick question about rsync configuration. If I just simply use ./configure, does "--iconv" supported by server? --disable-iconv disable rsync's --iconv option Thanks. -- Daniel -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubs

Re: Rsync Over SSH (Windows XP)

2009-03-14 Thread Daniel.Li
On Sat, 2009-03-14 at 23:41 +1100, David Overton wrote: > Hi Henri, > > 2009/3/14 henri : > > In essence I would like to determine if rsync a good choice when it comes to > > the backup of files on a Windows XP machine? > > > > Suggestions such as the following will all be warmly welcomed : > > >

RE: How to speed up rsync when haveing lots of files

2009-03-04 Thread Daniel.Li
On Wed, 2009-03-04 at 13:41 +0100, Boniforti Flavio wrote: > > This question has been answered. Upgrade to 3.0.5 on both sides. > > Why has upgrading to 3.0.5 to be done *on both sides*? Any explanation?! > As I'm running Debian Lenny (which has 3.0.3), do I have to switch to > Debian Sid to get i

Re: How to speed up rsync when haveing lots of files

2009-03-04 Thread Daniel.Li
On Tue, 2009-03-03 at 23:46 -0700, lewis butler wrote: > On 3-Mar-2009, at 22:55, Daniel.Li wrote: > > -z is apparently affect the performance when CPU has a lower > > frequency, > > like 200MHz or so. When doing rsync, 100% cpu occupied, which limits > > network perfo

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 21:26 -0700, lewis butler wrote: > On 3-Mar-2009, at 20:33, Daniel.Li wrote: > > On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: > >> On 3/3/2009 8:52 AM, m...@bortal.de wrote: > >>>>> unfortunatelly rsync is beeing REALLY slow and

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: > On 3/3/2009 8:52 AM, m...@bortal.de wrote: > >>> unfortunatelly rsync is beeing REALLY slow and produces a high load when > >>> we try to sync lots of files (>250 000 small files). > > >> What version of rsync are you using? > > > # rsync

Re: rsycing very small files

2009-02-27 Thread Daniel.Li
1. On Fri, 2009-02-27 at 08:20 -0500, Mag Gam wrote: > it works. But takes hours to do it. Was wondering if there was a faster way How much speed do u get to backup these files? Average? > > On Fri, Feb 27, 2009 at 8:04 AM, Paul Slootman wrote: > > On Fri 27 Feb 2009, Mag Gam wrote: > > > >

Time slot back schema

2009-02-23 Thread Daniel.Li
Hi List, Does Rsync support time slot back schema? E.G. (same local folder and remote destination) 1) first backup at 2008-12-24 2) second backup at 2009-02-12 But something bad happens to 2209-02-12, I need to restore files to first backup. Is there any way to do so? Any help is mostly appreci