Re: how to know the detail about a partial transfer?
On Thu 24 Jan 2008, Matt McCutchen wrote: is it possible to change those error messages into a standard pattern? Yes, it would be possible, but it would require going through and reformatting each of the 77 FERROR_XFER messages individually and would make a few of them less readable (e.g., link A = B failed would become B: link from A failed). Another option would be to print an additional failed to propagate B message after a failure of any kind on file B, and your script could just match those messages. Another approach is to prepend each error message with some type of code, e.g. link A = B failed becomes E0023: link A = B failed I remember from my days working with AIX that every tool did that, and while it looks a bit cluttered, it's a lot simpler to search the docs for the error code. And processing / detecting errors by scripts becomes very simple. Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--partial and --delete
Hi all, I need to use --partial (without using --partial-dir if possible). If --delete is also specified, does this cancel out the effect of --partial (since any previous partials are deleted before the transfers?) .. can;t seem to figure it out from the man pages alone... If this is so, will using --delete-after be sufficient to get the desired effect? Thanks, Julian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync on multiple ports?
Hi all, I need my rsync to listen on port 8090 as well as on the standard rsync port. Is this possible, and if so, how does one do this? Thanks! Robert -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsyncd performance when handling multiple clients in parallel
Hi, On Fri, 25 Jan 2008, chuang liu wrote: I use rsync to transfer multiple files from several clients to a server in parallel. I am wondering how many concurrent connections the server should handle to maximize the throughput (number of bytes written to server). In an extreme case, if only one connection allowed, the disk IO speed of server will not be fully utilized. On the other hand, if the server allows too many connections, the hard driver header will move around for writing different files, which may also slow down the disk IO throughput. Does rsyncd do any optimization for handling concurrent connections? and does anyone have experience on fine tuning the server to maximize the throughput? Thanks. The best fine tuning for rsync servers is to add more RAM. At ftp5.gwdg.de, I have 32 GB RAM and rsyncd is allowed to run 333 concurrent sessions (beneath 999 vsftpd and 2500 apache instances). These limits are not due to ressource limitation, but to limit race situations like f.e. a scripted rsync loop every five seconds. With 80 Mbytes/sec net output the disk input is mostly below 20 MByte/sec (varying with the diversity of the file requests) due to the buffer cache and long living inode cache, and all process memory can be held in core without swapping. Your base situation differs - you are doing input to the rsync server. So the buffer cache function is different for you (it will help mainly to order the disk writes more efficiently), but a big inode cache will save a lot of directory lookup disk I/O. A growing inode cache on 32-bit systems (say 700 MB) can lead to hash collisions which can almost starve the system, so I would recommend x86_64 arch for bigger rsync servers. Viele Grüße Eberhard Mönkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED]) -- Eberhard Mönkeberg Arbeitsgruppe IT-Infrastruktur E-Mail: [EMAIL PROTECTED] Tel.: +49 (0)551 201-1551 - Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Fassberg 11, 37077 Göttingen URL:http://www.gwdg.de E-Mail: [EMAIL PROTECTED] Tel.: +49 (0)551 201-1510Fax:+49 (0)551 201-2150 Geschäftsführer:Prof. Dr. Bernhard Neumair Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger Sitz der Gesellschaft: Göttingen Registergericht:Göttingen Handelsregister-Nr. B 598 --- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync more than 131072 files on linux
Hi all, Thanks for my RAM problem, it's OK now, Dell share me 32GB of RAM. But now when I try on my showroom with a lot of files in one directory I have this error: sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/* /mnt/destination/ -bash: /usr/bin/rsync: Argument list too long I ask to my best friend google and I have found a hard limitation (ARG_MAX) that you can know with: sho-lnx-001:~ # getconf ARG_MAX 131072 sho-lnx-001:~ # find /mnt/source/.* | wc -l 11756910 And I have read xarg with exec and chmod can be use for bypass ARG_MAX I try and don't understand what is wrong... but that doesn't want work fine... Can you help me please??? Thanks in advance... Sylvain GARGASSON Technicien EUDASYS 60/62, Rue du Maréchal Foch 78000 VERSAILLES Tél. : 01 39 25 66 66 Fax : 01 39 25 66 67 Céline Louis Assistante Tèl: +33 (0)1.39.25.66.79 Fax: +33 (0)1.39.25.66.67 Ce message contient des informations confidentielles couvertes par le secret professionnel. Si vous n'êtes pas le destinataire désigné, nous vous remercions de bien vouloir nous en aviser immédiatement et de nous retourner ce message ou de le détruire, sans faire un quelconque usage de son contenu, ni le communiquer ou le diffuser, ni en prendre aucune copie, électronique ou non. La sécurité des envois de messages électroniques ne peut être assurée. Ces messages peuvent notamment être interceptés, modifiés, altérés, détruits, perdus, arriver tardivement ou partiellement, ou contenir des virus. L'expéditeur ne saurait être tenu pour responsable des erreurs ou omissions qui résulteraient d'un envoi par message électronique. Si vous souhaitez vérifier l'authenticité du message et des fichiers joints, merci d'en solliciter une copie sur papier. image001.jpg-- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes
Hi Again, I reinstalled developer tools again on OS10.3.2 and ran make again and this time and saw more action but a lot of errors such as the one mentioned by Vitorio: error: `ACL_TYPE_DEFAULT' undeclared (first use in this function) Thanks, Rob D Here is the whole output from make: rsync 3.0.0pre8 configuration successful Robert-DuToits-Computer:~/rsync-3.0.0pre8 robertdutoit$ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c flist.c -o flist.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c rsync.c -o rsync.o rsync.c: In function `set_file_attrs': rsync.c:342: warning: unused parameter `fnamecmp' gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c generator.c -o generator.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c receiver.c -o receiver.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c cleanup.c -o cleanup.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c sender.c -o sender.o sender.c: In function `write_ndx_and_attrs': sender.c:150: warning: unused parameter `fname' sender.c:150: warning: unused parameter `file' gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c exclude.c -o exclude.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c util.c -o util.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c main.c -o main.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c checksum.c -o checksum.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c match.c -o match.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c syscall.c -o syscall.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c log.c -o log.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c backup.c -o backup.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c options.c -o options.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c io.c -o io.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c compat.c -o compat.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c hlink.c -o hlink.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c token.c -o token.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c uidlist.c -o uidlist.o uidlist.c: In function `match_uid': uidlist.c:210: warning: comparison between signed and unsigned uidlist.c: In function `match_gid': uidlist.c:222: warning: comparison between signed and unsigned uidlist.c:226: warning: comparison between signed and unsigned uidlist.c: In function `add_uid': uidlist.c:249: warning: comparison between signed and unsigned uidlist.c: In function `add_gid': uidlist.c:267: warning: comparison between signed and unsigned gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c socket.c -o socket.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c hashtable.c -o hashtable.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c fileio.c -o fileio.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c batch.c -o batch.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c clientname.c -o clientname.o clientname.c: In function `compare_addrinfo_sockaddr': clientname.c:269: warning: int format, size_t arg (arg 4) gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c chmod.c -o chmod.o gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c acls.c -o acls.o In file included from acls.c:23: lib/sysacls.h:288: error: parse error before the_acl lib/sysacls.h:289: error: parse error before entry_d lib/sysacls.h:290: error: parse error before entry lib/sysacls.h:291: error: parse error before sys_acl_get_file lib/sysacls.h:291: error: parse error before acl_type_t lib/sysacls.h:291: warning: type defaults to `int' in declaration of `sys_acl_get_file' lib/sysacls.h:291: warning: data definition has no type or storage class lib/sysacls.h:292: error: parse error before sys_acl_get_fd lib/sysacls.h:292: warning: type defaults to `int' in declaration of `sys_acl_get_fd' lib/sysacls.h:292: warning: data definition has no type or storage class lib/sysacls.h:293: error: parse error before sys_acl_init lib/sysacls.h:293: warning: type defaults to `int' in declaration of `sys_acl_init' lib/sysacls.h:293: warning: data definition has no type or storage class lib/sysacls.h:294: error: parse error before '*' token lib/sysacls.h:295: error: parse error before entry lib/sysacls.h:296: error: parse error before entry lib/sysacls.h:297: error: parse error before theacl lib/sysacls.h:298: error: parse error before acl_type_t lib/sysacls.h:299: error: parse error before acl_t lib/sysacls.h:301: error: parse error before the_acl acls.c:81: error: parse error before acl_t acls.c:81:
Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes
On Fri, 2008-01-25 at 21:38 -0500, Robert DuToit wrote: I reinstalled developer tools again on OS10.3.2 and ran make again and this time and saw more action but a lot of errors such as the one mentioned by Vitorio: error: `ACL_TYPE_DEFAULT' undeclared (first use in this function) Please try again with: ./configure --disable-acl-support --enable-xattr-support Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
On Friday 25 January 2008, Eberhard Moenkeberg wrote: On Fri, 25 Jan 2008, Sylvain Gargasson wrote: Thanks for my RAM problem, it's OK now, Dell share me 32GB of RAM. But now when I try on my showroom with a lot of files in one directory I have this error: sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/* /mnt/destination/ -bash: /usr/bin/rsync: Argument list too long It is just a bash isssue. bash is just passing the error up from the OS -mike signature.asc Description: This is a digitally signed message part. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
On Sat, 2008-01-26 at 05:36 +0100, Eberhard Moenkeberg wrote:
On Fri, 25 Jan 2008, Mike Frysinger wrote:
On Friday 25 January 2008, Eberhard Moenkeberg wrote:
On Fri, 25 Jan 2008, Sylvain Gargasson wrote:
sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/*
/mnt/destination/
-bash: /usr/bin/rsync: Argument list too long
It is just a bash isssue.
bash is just passing the error up from the OS
Wrong. Bash's command parameter buffer is too small.
No, the error is definitely coming from the OS. I ran the following
test on my Linux machine:
strace -f -o echo.strace bash -c '/bin/echo {1..100}' /dev/null
The output is here:
http://mattmccutchen.net/private/echo.strace
Note the execve system call near the end that fails with E2BIG.
Matt
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
Hi, On Fri, 25 Jan 2008, Mike Frysinger wrote: On Friday 25 January 2008, Eberhard Moenkeberg wrote: On Fri, 25 Jan 2008, Sylvain Gargasson wrote: Thanks for my RAM problem, it's OK now, Dell share me 32GB of RAM. But now when I try on my showroom with a lot of files in one directory I have this error: sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/* /mnt/destination/ -bash: /usr/bin/rsync: Argument list too long It is just a bash isssue. bash is just passing the error up from the OS Wrong. Bash's command parameter buffer is too small. Viele Grüße Eberhard Mönkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED])-- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
Hi,
On Fri, 25 Jan 2008, Matt McCutchen wrote:
On Sat, 2008-01-26 at 05:36 +0100, Eberhard Moenkeberg wrote:
On Fri, 25 Jan 2008, Mike Frysinger wrote:
On Friday 25 January 2008, Eberhard Moenkeberg wrote:
On Fri, 25 Jan 2008, Sylvain Gargasson wrote:
sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/*
/mnt/destination/
-bash: /usr/bin/rsync: Argument list too long
It is just a bash isssue.
bash is just passing the error up from the OS
Wrong. Bash's command parameter buffer is too small.
No, the error is definitely coming from the OS. I ran the following
test on my Linux machine:
strace -f -o echo.strace bash -c '/bin/echo {1..100}' /dev/null
The output is here:
http://mattmccutchen.net/private/echo.strace
Note the execve system call near the end that fails with E2BIG.
20250 write(2, bash: /bin/echo: Argument list t..., 40) = 40
20250 exit_group(126) = ?
This case is as clear as possible. Just leave the * and bash is out of
business.
Viele Grüße
Eberhard Mönkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED])--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Making rsync compile under Mac OS X 10.3.9 with extended attributes
Hi all, Somewhat along the same lines, I wanted to see if rsync 3 would work on Mac OS10.3X so I made a test partition and installed 10.3.2 on it ( no updates around). I then installed xcode 1.5 and with a fresh copy of rsync 3.0.0pre8 and tried ./configure which worked fine. When I tried make however I got the following error: Robert-DuToits-Computer:~/rsync-3.0.0pre8 robertdutoit$ make perl ./mkproto.pl ./*.c ./lib/compat.c Failed to create rounding.h! make: *** [rounding.h] Error 1 Does anyone have an idea how to make this work or if anyone has had success with OS10.3X and rsync 3? Thanks, Rob DuToit On Jan 25, 2008, at 11:45 AM, Vitorio Machado wrote: Le 24 janv. 08 à 16:30, Matt McCutchen a écrit : On Thu, 2008-01-24 at 13:25 +0100, Vitorio Machado wrote: Somebody knows if it's possible to compile rsync (version 3 would be great, but can be 2.6.x) under Mac OS X 10.3.9 with extended attributes? I have tried to compile rsync from source but the problem I found is that the xattr patch seems to need the acl patch. But 10.3 doesn't have the ACL support so doesn't provides the expected headers and the make command fails. In rsync 3, support for preserving acls and xattrs is in the trunk and the acls and xattrs patches just add acl and xattr protocol compatibility with older versions of rsync. Thus, you probably don't need to apply either patch. Try compiling rsync 3.0.0pre8 without patches, and if you still run into problems, post the exact error message(s). Matt First of all, thanks for your interest in my problem. I will post precise error repport next week, I'm not at work at the moment. The problem is that apparently xattr code depends on ACL code. But Mac OS X 10.3.9 doesn't have the needed API (gcc doesn't find headers about ACL code in the system). On a fresh downloaded source of rsync 3.0.0pre8 I've tried: 1) ./configure - configuration completes fine without error make - error on acl header as described (will report the precise error next week) 2) ./configure --disable-acl-support - configuration completes without error make - compilation goes just fine ./rsync --version - indicates in capabilities NO ACL and NO xattr support, so disabling ACL I've also disabled xattr (what I do need because the purpose is to backup mac files that are rich on ressource forks...) 3) ./configure --disable-acl-support --enable-xattr-support - configuration completes without error make - same acl error as in point number 1. This made me conclude the xattr code automatically activates the ACL support. And as ACL doesn't compile on 10.3 it seems it will not possible to have xattr on 10.3. Please say me that I'm wrong and that there's a way to trick it. Perhaps a patch like the one of RsyncX, http://www.quesera.com/reynhout/misc/rsync+hfsmode/ , or http://home.onthenet.com.au/~q/rsync/ , but applied to the last release. Note that I don't care so much about compability of the 10.3 rsync I'll get and other versions. The 10.3 rsync I want to compile doesn't need to talk with a rsync server, the backup will be done locally on a HFS+ volume. All I want is a fast incremental algorithm that preserve all the information (data and ressource fork basically, permissions, dates, etc would be fine but aren't mandatory) while making a copy. I can't use RsyncX version of rsync because it have an annoying double free bug. The version found at http://home.onthenet.com.au/~q/rsync/ is compiled 10.4+ only. I didn't tried to compile it on 10.3, will try it next week. http://www.quesera.com/reynhout/misc/rsync +hfsmode/ patch approach doesn't seem the good one for me, I don't want my files to be encoded as AppleDouble. And all those patch aren't official. It would be a lot easier to track bugs and get it progressing if it was the official rsync way to handle xattr. Don't you think it's the better approach?-- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
Hi, On Fri, 25 Jan 2008, Sylvain Gargasson wrote: Thanks for my RAM problem, it's OK now, Dell share me 32GB of RAM. But now when I try on my showroom with a lot of files in one directory I have this error: sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/* /mnt/destination/ -bash: /usr/bin/rsync: Argument list too long It is just a bash isssue. You can say rsync -av --progress --stats /mnt/source/ /mnt/destination/ just as well, avoiding this limitation. Viele Grüße Eberhard Mönkeberg ([EMAIL PROTECTED], [EMAIL PROTECTED])-- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --partial and --delete
Thanks Paul, understood. - Original Message - From: Paul Slootman [EMAIL PROTECTED] To: rsync@lists.samba.org Sent: Friday, January 25, 2008 5:32 PM Subject: Re: --partial and --delete On Fri 25 Jan 2008, Julian Pace Ross wrote: I need to use --partial (without using --partial-dir if possible). If --delete is also specified, does this cancel out the effect of --partial (since any previous partials are deleted before the transfers?) .. can;t seem to figure it out from the man pages alone... --partial means that that part of the file that was transferred gets renamed to the real thing when the transfer is interrupted. Hence it won't be deleted as the filename is one that is wanted. Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync on multiple ports?
On 25.01.2008 16:09, Matt McCutchen wrote: On Fri, 2008-01-25 at 12:26 -0800, Robert Denton wrote: I need my rsync to listen on port 8090 as well as on the standard rsync port. Is this possible, and if so, how does one do this? A single background daemon can listen on only one port. Just start two separate daemons with configuration files that are identical except for the port and pid file settings. A max connections limit will cover the total of the two daemons provided that both use the same lock file. Alternatively, if you use xinetd, you can configure two xinetd services (one for each port) that both point to the same daemon; this way you need only one daemon configuration file. Or you misuse the firewall, or use a port-forwarding program. The firewall misuse goes like this: iptables -t nat -A OUTPUT -p tcp --dport 8090 -j DNAT --to-destination 127.0.0.1:873 The firewall misuse obviously needs some kernel support in the form of netfilter with connection tracking and NATing. But i guess most distribution kernels should contain them. I personally misuse the firewall, in cooperation with some SSH-tunnels, to abstract away networking details when i use my [EMAIL PROTECTED] in different LANs. That works like a charm. :-) Bis denn -- Real Programmers consider what you see is what you get to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a you asked for it, you got it text editor -- complicated, cryptic, powerful, unforgiving, dangerous. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsyncd performance when handling multiple clients in parallel
Hi: I use rsync to transfer multiple files from several clients to a server in parallel. I am wondering how many concurrent connections the server should handle to maximize the throughput (number of bytes written to server). In an extreme case, if only one connection allowed, the disk IO speed of server will not be fully utilized. On the other hand, if the server allows too many connections, the hard driver header will move around for writing different files, which may also slow down the disk IO throughput. Does rsyncd do any optimization for handling concurrent connections? and does anyone have experience on fine tuning the server to maximize the throughput? Thanks. Chuang -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --partial and --delete
On Fri 25 Jan 2008, Julian Pace Ross wrote: I need to use --partial (without using --partial-dir if possible). If --delete is also specified, does this cancel out the effect of --partial (since any previous partials are deleted before the transfers?) .. can;t seem to figure it out from the man pages alone... --partial means that that part of the file that was transferred gets renamed to the real thing when the transfer is interrupted. Hence it won't be deleted as the filename is one that is wanted. Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync 3.0.0pre8 and Mac OS X
On Thu, 2008-01-24 at 10:11 -0500, Matt McCutchen wrote: So just pass --iconv=utf8mac,iso885915 when the Mac is sending and --iconv=iso885915,utf8mac when it is receiving, and the problem should go away. Just so the above doesn't confuse people in the future: I thought incorrectly that the --iconv argument had the form SRC,DEST . According to the man page the form is actually CLIENT,SERVER , which is different for a pull. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync more than 131072 files on linux
On Fri, 2008-01-25 at 19:03 +0100, Sylvain Gargasson wrote: Thanks for my RAM problem, it’s OK now, Dell share me 32GB of RAM. But now when I try on my showroom with a lot of files in one directory I have this error: sho-lnx-001:~ # rsync -av --progress --stats /mnt/source/* /mnt/destination/ -bash: /usr/bin/rsync: Argument list too long Remove the * from /mnt/source/* . Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync on multiple ports?
On Fri, 2008-01-25 at 12:26 -0800, Robert Denton wrote: I need my rsync to listen on port 8090 as well as on the standard rsync port. Is this possible, and if so, how does one do this? A single background daemon can listen on only one port. Just start two separate daemons with configuration files that are identical except for the port and pid file settings. A max connections limit will cover the total of the two daemons provided that both use the same lock file. Alternatively, if you use xinetd, you can configure two xinetd services (one for each port) that both point to the same daemon; this way you need only one daemon configuration file. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsyncd performance when handling multiple clients in parallel
On Fri, 2008-01-25 at 14:21 -0600, chuang liu wrote: Does rsyncd do any optimization for handling concurrent connections? No. Concurrent connections are handled by completely independent server processes. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[PATCH] Mention iconv --list in the man page.
--- Adding a hint about iconv --list is a good idea. This patch does it. Matt rsync.yo |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/rsync.yo b/rsync.yo index 047e360..870993a 100644 --- a/rsync.yo +++ b/rsync.yo @@ -2015,7 +2015,8 @@ dit(bf(--iconv=CONVERT_SPEC)) Rsync can convert filenames between character sets using this option. Using a CONVERT_SPEC of . tells rsync to look up the default character-set via the locale setting. Alternately, you can fully specify what conversion to do by giving a local and a remote charset -separated by a comma (local first), e.g. bf(--iconv=utf8,iso88591). +separated by a comma (local first), e.g. bf(--iconv=utf8,iso88591). (Run +iconv --list to see a list of the charset names that a machine supports.) Finally, you can specify a CONVERT_SPEC of - to turn off any conversion. The default setting of this option is site-specific, and can also be affected via the RSYNC_ICONV environment variable. -- 1.5.4.rc3.15.g4bbc -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: how to know the detail about a partial transfer?
On Fri, 2008-01-25 at 11:34 +0100, Paul Slootman wrote: On Thu 24 Jan 2008, Matt McCutchen wrote: is it possible to change those error messages into a standard pattern? Yes, it would be possible, but it would require going through and reformatting each of the 77 FERROR_XFER messages individually and would make a few of them less readable (e.g., link A = B failed would become B: link from A failed). Another option would be to print an additional failed to propagate B message after a failure of any kind on file B, and your script could just match those messages. Another approach is to prepend each error message with some type of code, e.g. link A = B failed becomes E0023: link A = B failed I remember from my days working with AIX that every tool did that, and while it looks a bit cluttered, it's a lot simpler to search the docs for the error code. And processing / detecting errors by scripts becomes very simple. you are right. that is what i am doing right now. prefix each message with all special prefix, easier to grep. Paul Slootman -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
