Error in Rsync.. Value too large
Hi... I am encountering the following error while syncing the data error message is: "send_files failed to open abcdata1/PABCBTABD.container000 : Value too large to be stored in data type" Size of the files for which the error is coming are: 31 GB, 35 GB, 40 GB, 52 GB, and 59 GB each. Please help I will be highly thankful to you. Thanks & Regards Nitin Agarwal
AW: Largest file system being synced
Hi Jason, > Are there any Linux users out there using the likes of > RAID'ed-NBD, CODA or > Intermezzo for a similar effect? > The NBD (network block device) looks interesting, it allows > you to mount a remote raw partition - so you can effectively > RAID over the network. I'd recomend drbd over a nbd + raid solution: * drbd knows to read from local device only so performance is generaly better * it deals nicely with disconnect / reconnect scenarios. I've used drbd in real-life configurations with linux FailSafe and with heartbeat as cluster managers, works quite nicely once you get it running. Problems with drbd: * current version doesn't like SMP * works at block level, so speed for a full synchronisation depends on filesystem size, not just used space. * resync speed limited to about 4MB/sec - limit aplies only for resync, not for normal read/write access. have a look at http://www.linbit.com/en/ for more information. Bye, Martin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Largest file system being synced
On Thu, Jun 27, 2002 at 11:30:39AM -0400, Granzow, Doug (NCI) wrote: > > I am currently syncing 1.3 terabytes of files using rsync. This is spread > across about 12 filesystems all on the same server. Unfortunately we are > planning to move away from rsync because it is taking too long to run and it > takes up too much memory (some of the rsync processes take up 1.5 GB of RAM What tool are you moving to? Did you use rsync for any databases. My company is considering using rsync to mirror some databases. Thanks, -- Zoong Pham[EMAIL PROTECTED] UNIX Systems AdministratorMercy Health and Aged Care Inc. To get my PGP public key, email me with the subject: Request for Zoong Pham's PGP public key -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync 2.5.5 and Mac OS X
I've got mine setup in inetd.conf to be executed on a per-call request of port 873 as written in the man pages. The advantage of using this method, is that when the client is done and disconnects, the daemon quits, thereby freeing up resources that it was using. Word of warning though if you want to try this route.. you need to add an rsync tcp entry for port 873 into Services with the NetInfo manager in addition to editing the /etc/inetd.conf file. You'll have to restart to get any changes to inetd.conf take effect as 'killall -HUP inetd' has no effect. When run in the manner above, it is started up as the root user (as specified in inetd.conf). - Original Message - From: Catalino Cuadrado <[EMAIL PROTECTED]> Date: Thursday, June 27, 2002 2:19 pm Subject: Re: rsync digest, Vol 1 #778 - 11 msgs > I haven't gotten it to work successfully, I'm still struggling > with the > same setgroups error that you are. However, I can tell you what I have > tried. So far I have rsync configured to run in --daemon mode > using the > command: sudo rsync --daemon I can prompt it for a list of > modules, but > when I go to copy a module, I get the error @ERROR: setgroups failed. > I can copy locally, from one HD to another, but I can't get it to copy > over the network, even using the sudo rsync ipaddress:/Filename / > method.So it's been kind of aggrivating. I'm thinking it has to do > with the fact > that rsync needs to be run locally, and you're running it from a > remoteserver... If there's a way to trick the system into thinking > that the > rsync daemon is running as root on the local machine, that would > be ideal. > Unfortunately, I don't know enough about the UNIX side of things > to get > that far. > -Tito > > > Message: 5 > > Date: Thu, 27 Jun 2002 10:08:00 -0600 > > From: [EMAIL PROTECTED] > > Subject: rsync 2.5.5 and Mac OS X > > To: [EMAIL PROTECTED] > > Reply-To: [EMAIL PROTECTED] > > > > Greetings, > > > > Has anyone successfully compiled rsync 2.5.5 under Mac OS X (not > > Server), and actually have it work fully? > > > > I'm in a position where I'm trying to setup an automated nightly > remote> backup between and Cobalt Raq4 (running a variant of > Redhat Linux) and a > > Mac running OS X 10.1.5 with the latest development tools > installed. As > > has been pointed out previously on the list, one has to > ./configure with > > the --disable-ipv6 flag to even get it to compile at all. However > > there's further issues when trying to back up to the Mac running > rsync> in inetd --daemon mode. > > > > When I issue the following command: > > > > --- > > rsync -avz /home [EMAIL PROTECTED]::raqbackup > > --- > > > > I get the following error: > > > > --- > > @ERROR: setgroups failed > > rsync: connection unexpectedly closed (79 bytes read so far) > > rsync error: error in rsync protocol data stream (code 12) at > io.c(150)> --- > > > > On the server end, rsyncd.log shows the following: > > > > --- > > 2002/06/27 09:50:03 [19768] setgroups failed: Invalid argument > > --- > > > > I've had luck running rsync 2.5.2 running through ssh, but > because I > > can't get it to retain the user/group ownership settings it's > completely> useless as 'mirrored' backup unless I spent the next > month hand > > restoring the file ownership's by hand. The man pages noted this > > limitation, unless running as super-user, which I haven't been > able to > > get to work (use chroot=yes in rsyncd.conf doesn't seem to do it) > > > > If anyone has been able to get Mac OS X 10.1.5 running as a > rsync server > > in such a fashion, I'd really appreciate hearing from you. > > > > For debuging purposes, my current rsyncd.conf is: > > > > --- > > use chroot = yes > > max connections = 1 > > syslog facility = local5 > > motd file = /etc/rsyncd.motd > > log file = /var/log/rsyncd.log > > pid file = /var/run/rsyncd.pid > > lock file = /var/run/rsync.lock > > > > [raqbackup] > > path = /Volumes/Eeyore/RaqBackup > > comment = Raq Backup Directory > > auth users = xxx > > secrets file = /etc/rsyncd.secrets > > --- > > > > and the destination directory is set as: > > > > --- > > drwxrwxrwx2 nobody nobody 24 Jun 26 22:09 RaqBackup > > --- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
cygwin rsync and ascii files
I am ysyncing text files between NT/95/XP machines and Solaris. I noted that the text/ascii files created on the windows platforms contain ^M at the end of each line when transfered to the Solaris system. This can be explained by the binary transmission and can use FTP is ASCII mode to prevent this from happening. It is crutial that we preserve the time stamp, so running dos2unix isn't one of the top solutions (unless I write something to touch the file back to the "original" time). Is there an option to rsync files in ASCII mode? I would also appreciate and entertain other solutions. Thanks in advance, Mike -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Largest file system being synced
On Thu, Jun 27, 2002 at 01:31:12PM -0700, jw schultz wrote: > It doesn't work at the filesystem level. It works at the > block device level. Every time a block is modified it is > queued for transmission to the mirror(s). If the same block Are there any Linux users out there using the likes of RAID'ed-NBD, CODA or Intermezzo for a similar effect? The NBD (network block device) looks interesting, it allows you to mount a remote raw partition - so you can effectively RAID over the network. Supports transaction logs too (which would be necessary in a rsync-style role) -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Largest file system being synced
On Thu, Jun 27, 2002 at 02:21:08PM -0400, Granzow, Doug (NCI) wrote: > We're planning to move to Veritas Volume Replicator. It has the advantage > of working at the filesystem level, so whenever a write is done on the > primary site, the same write is automatically done on the mirror site. For > what I am doing here, it should (hopefully!) work a lot better than rsync > because it runs continuously, and it doesn't have the startup overhead of > building a file list, which is the biggest problem with rsync when you are > dealing with large groups of files. Unless this is a new product I've used it. It works fairly well and unlike most Veritas products the error messages actually mean something. It has been about two years so this could be a new product but the one i dealt with was really meant to be used as part of a N second fail-over solution with heartbeats et al. I give my comments here to further general knowledge and so you can hear from someone other than a Veritas spokes-being. It doesn't work at the filesystem level. It works at the block device level. Every time a block is modified it is queued for transmission to the mirror(s). If the same block is queued twice there is no coalescence, it will be sent twice. We did it over a VPN and Oracle kept sending the same block over and over every five seconds due to an acknowledged Oracle bug. There are three caveats to be aware of. You will get more network traffic than you probably expected. Only one end of the connection can be used at a time. And the filesystem on the mirror(s) is in the same state it would be if you pulled the plug so to go on-line requires the filesystem to replay the journal. It served the purpose, over a VPN needed more babysitting than i liked, but it did work fairly well but i disagree with the approach. > > > > > -Original Message- > > From: Breedlove, Robert [mailto:[EMAIL PROTECTED]] > > Sent: Thursday, June 27, 2002 11:46 AM > > To: Granzow, Doug (NCI); '[EMAIL PROTECTED]' > > Subject: RE: Largest file system being synced > > > > > > What are you moving to? > > > > -Original Message- > > From: Granzow, Doug (NCI) [mailto:[EMAIL PROTECTED]] > > Sent: Thursday, June 27, 2002 8:31 AM > > To: '[EMAIL PROTECTED]' > > Subject: RE: Largest file system being synced > > > > > > > > I am currently syncing 1.3 terabytes of files using rsync. > > This is spread > > across about 12 filesystems all on the same server. > > Unfortunately we are > > planning to move away from rsync because it is taking too > > long to run and it > > takes up too much memory (some of the rsync processes take up > > 1.5 GB of RAM > > -- if we get two of those running at once, the server dies). > > We have been > > very happy with rsync but it has recently reached a critical > > mass where it > > can no longer handle the number of files we are trying to > > sync in a timely > > manner. But if you are looking for the largest rsync site, > > we might be a > > contender. :) > > > > FYI, we also use (and plan to continue to use) rsync for > > several smaller > > mirroring operations. > > > > -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync digest, Vol 1 #778 - 11 msgs
I haven't gotten it to work successfully, I'm still struggling with the same setgroups error that you are. However, I can tell you what I have tried. So far I have rsync configured to run in --daemon mode using the command: sudo rsync --daemon I can prompt it for a list of modules, but when I go to copy a module, I get the error @ERROR: setgroups failed. I can copy locally, from one HD to another, but I can't get it to copy over the network, even using the sudo rsync ipaddress:/Filename / method. So it's been kind of aggrivating. I'm thinking it has to do with the fact that rsync needs to be run locally, and you're running it from a remote server... If there's a way to trick the system into thinking that the rsync daemon is running as root on the local machine, that would be ideal. Unfortunately, I don't know enough about the UNIX side of things to get that far. -Tito > Message: 5 > Date: Thu, 27 Jun 2002 10:08:00 -0600 > From: [EMAIL PROTECTED] > Subject: rsync 2.5.5 and Mac OS X > To: [EMAIL PROTECTED] > Reply-To: [EMAIL PROTECTED] > > Greetings, > > Has anyone successfully compiled rsync 2.5.5 under Mac OS X (not > Server), and actually have it work fully? > > I'm in a position where I'm trying to setup an automated nightly remote > backup between and Cobalt Raq4 (running a variant of Redhat Linux) and a > Mac running OS X 10.1.5 with the latest development tools installed. As > has been pointed out previously on the list, one has to ./configure with > the --disable-ipv6 flag to even get it to compile at all. However > there's further issues when trying to back up to the Mac running rsync > in inetd --daemon mode. > > When I issue the following command: > > --- > rsync -avz /home [EMAIL PROTECTED]::raqbackup > --- > > I get the following error: > > --- > @ERROR: setgroups failed > rsync: connection unexpectedly closed (79 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(150) > --- > > On the server end, rsyncd.log shows the following: > > --- > 2002/06/27 09:50:03 [19768] setgroups failed: Invalid argument > --- > > I've had luck running rsync 2.5.2 running through ssh, but because I > can't get it to retain the user/group ownership settings it's completely > useless as 'mirrored' backup unless I spent the next month hand > restoring the file ownership's by hand. The man pages noted this > limitation, unless running as super-user, which I haven't been able to > get to work (use chroot=yes in rsyncd.conf doesn't seem to do it) > > If anyone has been able to get Mac OS X 10.1.5 running as a rsync server > in such a fashion, I'd really appreciate hearing from you. > > For debuging purposes, my current rsyncd.conf is: > > --- > use chroot = yes > max connections = 1 > syslog facility = local5 > motd file = /etc/rsyncd.motd > log file = /var/log/rsyncd.log > pid file = /var/run/rsyncd.pid > lock file = /var/run/rsync.lock > > [raqbackup] > path = /Volumes/Eeyore/RaqBackup > comment = Raq Backup Directory > auth users = xxx > secrets file = /etc/rsyncd.secrets > --- > > and the destination directory is set as: > > --- > drwxrwxrwx2 nobody nobody 24 Jun 26 22:09 RaqBackup > --- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: extension of rsync on crypted files
On Thu, Jun 27, 2002 at 03:26:51PM +0200, Mikael moshir wrote: > > Hello, > > I am a french student and I have written a technical > report on an extension of the rsync algorithm to crypted > files. I started from the situation of a client machine A > user who doesn't wish to save an original file v0 and its > successive versions v1 v2 v3 ... on a distant server B but > rather to save the private ciphering of these files on the > server. Let C be the cipher algorithm and xi=C(vi) the > ciphering of the clear file vi, the client A wishes to > save x0, x1, x2... on the server B and not v0,v1 This > situation arrises for secured files backup providers which > offer an option of a secured and confidential file's > history backup - a client can keep the history of a file. > > Trying to use rsync here is not a good solution because > the ciphering transformation on file versions disperses > the correlations between them and then when Rsync tries to > localize common blocks between ciphered versions, he > hardly ever finds and the compression resulting from the > common sequences factorisation can't happen. > > My method proposes a solution to this problem and allows > to save ciphered versions on a distant server with costs > (storage, communications, algorithmic complexity) > comparable to rsync. I want to know if anyone finds my > work of interest and if anyone of you knows if such a > problem have been adressed before. I keep the > documentation of my article (in french but i am working on > an english translation) for anyone who would have more > details. Thank you! > > Ps: sorry for my small english skill. That skill is sufficient but you need shorter lines (hit the carriage return). I will try to clarify since it looks like this hasn't been understood. terminology: plaintext == unencrypted file ciphertext == encrypted file client == local system server == distant system You have a client with plaintext files. The server provides access to multiple versions of files but on the server they are encrypted. As you have described it this sounds a great deal like a SCM (source code management) system with encryption thrown in. Presumably the encryption is because the user doesn't wish to trust the owner of the server with maintaining his privacy. Judging by several recent threads there are quite a few people who could be interested. I suggest you look in the list archives for the word "encrypt". You speak of a technical report that i assume actually discusses the method. If you can distill that down (cut out the academic verbiage and focus on actual method sans proofs) to a couple of hundred words or less you could post it for discussion. A link to source code wouldn't hurt. We focus here on what actually works and even broken code speaks louder than speculation. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
RE: Largest file system being synced
We're planning to move to Veritas Volume Replicator. It has the advantage of working at the filesystem level, so whenever a write is done on the primary site, the same write is automatically done on the mirror site. For what I am doing here, it should (hopefully!) work a lot better than rsync because it runs continuously, and it doesn't have the startup overhead of building a file list, which is the biggest problem with rsync when you are dealing with large groups of files. > -Original Message- > From: Breedlove, Robert [mailto:[EMAIL PROTECTED]] > Sent: Thursday, June 27, 2002 11:46 AM > To: Granzow, Doug (NCI); '[EMAIL PROTECTED]' > Subject: RE: Largest file system being synced > > > What are you moving to? > > -Original Message- > From: Granzow, Doug (NCI) [mailto:[EMAIL PROTECTED]] > Sent: Thursday, June 27, 2002 8:31 AM > To: '[EMAIL PROTECTED]' > Subject: RE: Largest file system being synced > > > > I am currently syncing 1.3 terabytes of files using rsync. > This is spread > across about 12 filesystems all on the same server. > Unfortunately we are > planning to move away from rsync because it is taking too > long to run and it > takes up too much memory (some of the rsync processes take up > 1.5 GB of RAM > -- if we get two of those running at once, the server dies). > We have been > very happy with rsync but it has recently reached a critical > mass where it > can no longer handle the number of files we are trying to > sync in a timely > manner. But if you are looking for the largest rsync site, > we might be a > contender. :) > > FYI, we also use (and plan to continue to use) rsync for > several smaller > mirroring operations. > > > > I'm interested in large file system replication capability of > > rsync. Could > > some of you people who use it share how large their mirroring > > is? What would > > you say is the largest sized site being mirrored using rsync? > > > > Thanks! > > > > JP -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync finishes but incomplete
You'll have to be more specific if you want some help. The best thing to do would be to try to create a simple example that you can fully explain so somebody else could reproduce it. Very often that helps people to understand the root cause just by going through the exercise. At the least you'll need to exactly specify the rsync versions, the cygwin versions, which files weren't copied, and any error messages that are going to stderr. It doesn't look like you're directing stderr anywhere, and on most systems that will get sent by email to the owner of the account that cron is running under. Rather than redirecting job.log explicitly on every command, you might want to surround the whole thing with parentheses and add "2>&1", like this: ( ... your commands without the ... ) >> /home/rsync/job.log 2>&1 - Dave Dykstra On Tue, Jun 25, 2002 at 08:59:14AM -0400, J. Davis wrote: > Hello, > > I have a remote win2k machine and local NT4 machine running rsync over ssh, by way >of cygwin. > I have a cron script that grabs a directory from remote and puts it on > local. It' very simple > > #!/bin/bash > > echo "-" >> /home/rsync/job.log > echo "Running rsync job" >> /home/rsync/job.log > gnudate >> /home/rsync/job.log > rsync -az --stats -e ssh --exclude "WIN386.SWP" --exclude "Cache/" --exclude "*.AVB" >--exclude "APPS/" --exclude "virus update/" [EMAIL PROTECTED]:/cygdrive/e/data >/cygdrive/g/901/>>/home/rsync/job.log > echo "Done!" >> /home/rsync/job.log > gnudate >> /home/rsync/job.log > echo "-" >> /home/rsync/job.log > > It used to always hang but then I removed the -v option after reading > about that in the archives. > So according to my job.log the job finishes, however, it's apparent that > it has left a great many files on the remote system uncopied. Files that don't > match my --exclude args. > I'm at a loss for how to trouble shoot this one. > > -Jake -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: documentation bug for --daemon "use chroot" in conjunction with-o and -g
You da boss. And thanks for looking into this, it'll save other folks heartache in the future. Scott Dave Dykstra wrote: >On Tue, Jun 25, 2002 at 05:11:40PM -0400, pyxl wrote: > >>Well, >> >>Looking at the source for 2.5.5, I'm only seeing mention of it under the >>--numeric-ids flag in rsync(1); --owner doesn't make mention of it at all. >>What you quoted must be in CVS (I'd look, but I don't know how to do a >>remote CVS checkout yet, and am too lazy to learn at this moment). >> > >I researched it at >http://cvs.samba.org/cgi-bin/cvsweb/rsync/rsync.yo > >and see that it was taken out of the --owner item in rsync 2.5.1 by change >1.86: >http://cvs.samba.org/cgi-bin/cvsweb/rsync/rsync.yo.diff?r1=1.85&r2=1.86 > >That patch had many wording changes to the man page, suggested by somebody >from the community, and I think it was wrong to drop this info. I will >fix it for the next release, but I don't want to use your suggested wording >because I don't want to make it sound like we're recommending that people >do "use chroot = no"; instead, I will just inform them of the consequences. > >- Dave Dykstra > > > >>Here's a patch for the changes to those man pages against the 2.5.5 release >>tree. It changes --owner in rsync(1) and "use chroot" in rsyncd.conf(5). >> >>diff -uNr rsync-2.5.5/rsync.1 rsync-2.5.5_update/rsync.1 >>--- rsync-2.5.5/rsync.1 Wed Feb 6 16:21:19 2002 >>+++ rsync-2.5.5_update/rsync.1Tue Jun 25 16:28:52 2002 >>@@ -488,7 +488,9 @@ >>.IP "\fB-o, --owner\fP" >>This option causes rsync to set the owner of the >>destination file to be the same as the source file\&. On most systems, >>-only the super-user can set file ownership\&. >>+only the super-user can set file ownership\&. For this to work when one >>+side is an rsync daemon, the module must be configured with "use chroot = >>no" >>+or else only numeric uid and gid will be provided\&. >>.IP >>.IP "\fB-g, --group\fP" >>This option causes rsync to set the group of the >>diff -uNr rsync-2.5.5/rsyncd.conf.5 rsync-2.5.5_update/rsyncd.conf.5 >>--- rsync-2.5.5/rsyncd.conf.5 Fri Aug 31 04:12:35 2001 >>+++ rsync-2.5.5_update/rsyncd.conf.5 Tue Jun 25 16:46:51 2002 >>@@ -144,7 +144,9 @@ >>of not being able to follow symbolic links outside of the new root path >>when reading\&. When "use chroot" is false, for security reasons >>symlinks may only be relative paths pointing to other files within the >>-root path, and leading slashes are removed from absolute paths\&. The >>+root path, and leading slashes are removed from absolute paths\&. Note >>+that for --owner (-o) and --group (-g) client side options to function, >>+"use chroot" must be set to false for the module\&. The >>default for "use chroot" is true\&. >>.IP >>.IP "\fBmax connections\fP" >> > > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: documentation bug for --daemon "use chroot" in conjunction with -o and -g
On Tue, Jun 25, 2002 at 05:11:40PM -0400, pyxl wrote: > Well, > > Looking at the source for 2.5.5, I'm only seeing mention of it under the > --numeric-ids flag in rsync(1); --owner doesn't make mention of it at all. > What you quoted must be in CVS (I'd look, but I don't know how to do a > remote CVS checkout yet, and am too lazy to learn at this moment). I researched it at http://cvs.samba.org/cgi-bin/cvsweb/rsync/rsync.yo and see that it was taken out of the --owner item in rsync 2.5.1 by change 1.86: http://cvs.samba.org/cgi-bin/cvsweb/rsync/rsync.yo.diff?r1=1.85&r2=1.86 That patch had many wording changes to the man page, suggested by somebody from the community, and I think it was wrong to drop this info. I will fix it for the next release, but I don't want to use your suggested wording because I don't want to make it sound like we're recommending that people do "use chroot = no"; instead, I will just inform them of the consequences. - Dave Dykstra > Here's a patch for the changes to those man pages against the 2.5.5 release > tree. It changes --owner in rsync(1) and "use chroot" in rsyncd.conf(5). > > diff -uNr rsync-2.5.5/rsync.1 rsync-2.5.5_update/rsync.1 > --- rsync-2.5.5/rsync.1 Wed Feb 6 16:21:19 2002 > +++ rsync-2.5.5_update/rsync.1Tue Jun 25 16:28:52 2002 > @@ -488,7 +488,9 @@ > .IP "\fB-o, --owner\fP" > This option causes rsync to set the owner of the > destination file to be the same as the source file\&. On most systems, > -only the super-user can set file ownership\&. > +only the super-user can set file ownership\&. For this to work when one > +side is an rsync daemon, the module must be configured with "use chroot = > no" > +or else only numeric uid and gid will be provided\&. > .IP > .IP "\fB-g, --group\fP" > This option causes rsync to set the group of the > diff -uNr rsync-2.5.5/rsyncd.conf.5 rsync-2.5.5_update/rsyncd.conf.5 > --- rsync-2.5.5/rsyncd.conf.5 Fri Aug 31 04:12:35 2001 > +++ rsync-2.5.5_update/rsyncd.conf.5 Tue Jun 25 16:46:51 2002 > @@ -144,7 +144,9 @@ > of not being able to follow symbolic links outside of the new root path > when reading\&. When "use chroot" is false, for security reasons > symlinks may only be relative paths pointing to other files within the > -root path, and leading slashes are removed from absolute paths\&. The > +root path, and leading slashes are removed from absolute paths\&. Note > +that for --owner (-o) and --group (-g) client side options to function, > +"use chroot" must be set to false for the module\&. The > default for "use chroot" is true\&. > .IP > .IP "\fBmax connections\fP" -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync 2.5.5 and Mac OS X
Greetings, Has anyone successfully compiled rsync 2.5.5 under Mac OS X (not Server), and actually have it work fully? I'm in a position where I'm trying to setup an automated nightly remote backup between and Cobalt Raq4 (running a variant of Redhat Linux) and a Mac running OS X 10.1.5 with the latest development tools installed. As has been pointed out previously on the list, one has to ./configure with the --disable-ipv6 flag to even get it to compile at all. However there's further issues when trying to back up to the Mac running rsync in inetd --daemon mode. When I issue the following command: --- rsync -avz /home [EMAIL PROTECTED]::raqbackup --- I get the following error: --- @ERROR: setgroups failed rsync: connection unexpectedly closed (79 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(150) --- On the server end, rsyncd.log shows the following: --- 2002/06/27 09:50:03 [19768] setgroups failed: Invalid argument --- I've had luck running rsync 2.5.2 running through ssh, but because I can't get it to retain the user/group ownership settings it's completely useless as 'mirrored' backup unless I spent the next month hand restoring the file ownership's by hand. The man pages noted this limitation, unless running as super-user, which I haven't been able to get to work (use chroot=yes in rsyncd.conf doesn't seem to do it) If anyone has been able to get Mac OS X 10.1.5 running as a rsync server in such a fashion, I'd really appreciate hearing from you. For debuging purposes, my current rsyncd.conf is: --- use chroot = yes max connections = 1 syslog facility = local5 motd file = /etc/rsyncd.motd log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock [raqbackup] path = /Volumes/Eeyore/RaqBackup comment = Raq Backup Directory auth users = xxx secrets file = /etc/rsyncd.secrets --- and the destination directory is set as: --- drwxrwxrwx2 nobody nobody 24 Jun 26 22:09 RaqBackup --- Thank you for any assistance anyone can offer. Cheers! -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
RE: Largest file system being synced
What are you moving to? -Original Message- From: Granzow, Doug (NCI) [mailto:[EMAIL PROTECTED]] Sent: Thursday, June 27, 2002 8:31 AM To: '[EMAIL PROTECTED]' Subject: RE: Largest file system being synced I am currently syncing 1.3 terabytes of files using rsync. This is spread across about 12 filesystems all on the same server. Unfortunately we are planning to move away from rsync because it is taking too long to run and it takes up too much memory (some of the rsync processes take up 1.5 GB of RAM -- if we get two of those running at once, the server dies). We have been very happy with rsync but it has recently reached a critical mass where it can no longer handle the number of files we are trying to sync in a timely manner. But if you are looking for the largest rsync site, we might be a contender. :) FYI, we also use (and plan to continue to use) rsync for several smaller mirroring operations. > I'm interested in large file system replication capability of > rsync. Could > some of you people who use it share how large their mirroring > is? What would > you say is the largest sized site being mirrored using rsync? > > Thanks! > > JP -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: extension of rsync on crypted files
> "MM" == mmikaelfr > wrote the following on Thu, 27 Jun 2002 15:26:51 +0200 (CEST) MM> I am a french student and I have written a technical report on MM> an extension of the rsync algorithm to crypted files. I started MM> from the situation of a client machine A user who doesn't wish MM> to save an original file v0 and its successive versions v1 v2 v3 MM> ... on a distant server B but rather to save the private MM> ciphering of these files on the server. Let C be the cipher MM> algorithm and xi=C(vi) the ciphering of the clear file vi, the MM> client A wishes to save x0, x1, x2... on the server B and not MM> v0,v1 This situation arrises for secured files backup MM> providers which offer an option of a secured and confidential MM> file's history backup - a client can keep the history of a file. Sounds interesting. Do you mean that the server holds multiple versions of the file so older versions can also be restored? Is this done by storing checksum information about older files on the client? Also, if I understood right, isn't this a bit different from rsync since on the server you wouldn't see the files x0, x1, x2, ... but rather the files x0, d1, d2, ... where d1 is a some kind of delta from x0 to x1 (or I suppose an encrypted delta from v0 to v1)? Or have I misunderstood your system? -- Ben Escoto msg04442/pgp0.pgp Description: PGP signature
RE: Largest file system being synced
I am currently syncing 1.3 terabytes of files using rsync. This is spread across about 12 filesystems all on the same server. Unfortunately we are planning to move away from rsync because it is taking too long to run and it takes up too much memory (some of the rsync processes take up 1.5 GB of RAM -- if we get two of those running at once, the server dies). We have been very happy with rsync but it has recently reached a critical mass where it can no longer handle the number of files we are trying to sync in a timely manner. But if you are looking for the largest rsync site, we might be a contender. :) FYI, we also use (and plan to continue to use) rsync for several smaller mirroring operations. > I'm interested in large file system replication capability of > rsync. Could > some of you people who use it share how large their mirroring > is? What would > you say is the largest sized site being mirrored using rsync? > > Thanks! > > JP -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
extension of rsync on crypted files
Hello, I am a french student and I have written a technical report on an extension of the rsync algorithm to crypted files.I started from the situation of a client machine A user who doesn't wish to save an original file v0 and its successive versions v1 v2 v3 ... on a distant server B but rather to save the private ciphering of these files on the server. Let C be the cipher algorithm and xi=C(vi) the ciphering of the clear file vi, the client A wishes to save x0, x1, x2... on the server B and not v0,v1 This situation arrises for secured files backup providers which offer an option of a secured and confidential file's history backup - a client can keep the history of a file. Trying to use rsync here is not a good solution because the ciphering transformation on file versions disperses the correlations between them and then when Rsync tries to localize common blocks between ciphered versions, he hardly ever finds and the compression resulting from the common sequences factorisation can't happen. My method proposes a solution to this problem and allows to save ciphered versions on a distant server with costs (storage, communications, algorithmic complexity) comparable to rsync.I want to know if anyone finds my work of interest and if anyone of you knows if such a problem have been adressed before. I keep the documentation of my article (in french but i am working on an english translation) for anyone who would have more details. Thank you! Ps: sorry for my small english skill.Yahoo! Mail -- Une adresse @yahoo.fr gratuite et en français !
