Re: [rsync] '-c' vs. '-I'
Read Wayne Davidson post sent 31:06 before yours. 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/07/2005 08:33 AM To rsync@lists.samba.org cc Subject [rsync] '-c' vs. '-I' What is the practical/functional difference between using '-c' (--checksum) & '-I' (--ignore-times), all other related options being equal ? Thanks. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Linux to Windows
In case cygwin was doing some filename mapping, I tried it in cmd: C:\DOCUME~1\ADMINI~1>cd t C:\DOCUME~1\ADMINI~1\t>dir Volume in drive C has no label. Volume Serial Number is 5CC0-9DEE Directory of C:\DOCUME~1\ADMINI~1\t 08/24/2005 08:54 AM . 08/24/2005 08:54 AM .. 0 File(s) 0 bytes 2 Dir(s) 28,321,488,896 bytes free C:\DOCUME~1\ADMINI~1\t>edit .dotfile C:\DOCUME~1\ADMINI~1\t>del .dotfile C:\DOCUME~1\ADMINI~1\t>dir Volume in drive C has no label. Volume Serial Number is 5CC0-9DEE Directory of C:\DOCUME~1\ADMINI~1\t 08/24/2005 08:54 AM . 08/24/2005 08:54 AM .. 0 File(s) 0 bytes 2 Dir(s) 28,321,488,896 bytes free C:\DOCUME~1\ADMINI~1\t>echo dot >.dotfile C:\DOCUME~1\ADMINI~1\t>dir Volume in drive C has no label. Volume Serial Number is 5CC0-9DEE Directory of C:\DOCUME~1\ADMINI~1\t 08/24/2005 08:55 AM . 08/24/2005 08:55 AM .. 08/24/2005 08:55 AM 6 .dotfile 1 File(s) 6 bytes 2 Dir(s) 28,321,488,896 bytes free C:\DOCUME~1\ADMINI~1\t>type .dotfile dot C:\DOCUME~1\ADMINI~1\t>del .dotfile C:\DOCUME~1\ADMINI~1\t>ver Microsoft Windows XP [Version 5.1.2600] C:\DOCUME~1\ADMINI~1\t> 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] "Brent Blayney" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 08/24/2005 06:40 AM To cc Subject Linux to Windows I'm sure, too. Apparently, you can't start a filename with a period in Win, either, as my previous examples showed. 1120817285.22306_0.mail (works fine) .1124450874.30945_0.mail (reports 0KB) .1123700716.P14142Q0M23.mail (reports 0KB) .1087907444.7006_1.mail.domain.com,U=1,W=42566 (reports 0KB) Is it possible for Cygwin to emulate an ext2 or ext3 partition on the NTFS drive? The other idea I had was to have a dual-bootsystem and actually back up when booted in Linux, but obviously this seriously complicates the automation process.> Yep. I'm sure. > There's no colon's in your filenames there. > Can't have these characters in a filename: > \ / : * ? " < > | > -john > David Filion wrote: > John Jablonski wrote: > >> Thing is, it's not an rsync problem. It's a windows filesystem >> problem. Or at least a windows problem of some sort. >> >> You can't have a file called: >> 1124816518.8634_2.mailbox:2,S > > Sure about that? > > 1115996480.12736_4.ritalin.autolinq.com,S=7399_2,S > > No problem on NTFS or fat32. These were extracted from a tar, not > directly created using rsync though. > -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: All the feature requests... A better way?
Need I remind you that the source code is freely-available? Implement the algorithm in perl, use that as a module, and build others around it. The world needs more heroes. 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync is flaky going to Penang
There are times when rsync just can't complete a task. I had a buggy network and was required to maintain a large distribution on dozens of buggy NAS devices all over the world. I eventually had to write a sort of distributed find/diff/rm/tar/untar system. 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 04/04/2005 04:41 PM To <[EMAIL PROTECTED]>, cc Subject rsync is flaky going to Penang Hello, We are experiencing flaky behavior from rsync when attempting to rsync directories/files to a server in Penang. Several times the job seems to ‘hang’ and never completes. Penang then is therefore missing a lot of required cad files. Have any of you experienced the same thing and what did you do to fix the problem? Does anyone know of a better tool to use? We chose rsync after rdist gave us problems. Perhaps we’re missing an important flag in our rsync commands but I think it’s set up correctly. Thanks and Best Regards, Jackie Wright Agilent Technologies - WSD R&D IT Engineer Telnet: 435-6653 or (408) 435-6653 Cell: (510) 825-7638 Fax: (408) 435-4803 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsyncd.conf without --daemon?
If you use -e ssh with :: or rsync:// syntax, it sshes over and starts and uses a private rsyncd in ssh tunnel. If you use it with single-colon syntax, it starts an rsync listener on the other end that obeys commands and passes data. 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] Philip Thompson <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/03/2005 09:29 AM Please respond to Philip Thompson To rsync@lists.samba.org cc Subject rsyncd.conf without --daemon? Hello all, I'm not a total newbie, but probably close enough. I have tried to find the answer to this question, but have been unsuccessful. I'm testing out rsync over ssh using empty passphrases between 2 OpenBSD 3.5 boxes using rsync 2.5.7 (yes i know i should upgrade, but can't because of BSD ports issues). Everything seems to be working, but I'm very confused as to why it is working. I haven't started rsync in daemon mode on the box I'm retrieving files from and there's nothing in /etc/inetd.conf either. I've placed an rsyncd.conf file in the directory of the rsync user and it seems to be reading the file. Is this normal? The documentation states that "The rsyncd.conf file is the runtime configuration file for rsync wen run as an rsync server." But as far as I can figure all I've done is installed the rsync client on both boxes. I guess my question is if rsyncd.conf is for daemon mode why/how is it reading my rsyncd.conf file? There is no rsync process running on my target server so I'm guessing that it's because I'm using SSH as the transport, but how does it know to use the rsyncd.conf file in that particular directory or at all for that matter? Happy it's working, but confused as to why. Thanks in advance!!! -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync huge tar files
If it is, as you say, uncompressed, rsync will work on it as-is, finding and sending the changes. 73, Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] Harald Dunkel <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 02/04/2005 02:37 AM To rsync@lists.samba.org cc Subject rsync huge tar files Hi folks, Are there any tricks known to let rsync operate on huge tar files? I've got a local tar file (e.g. 2GByte uncompressed) that is rebuilt each night (with just some tiny changes, of course), and I would like to update the remote copies of this file without extracting the tar files into temporary directories. Any ideas? Regards Harri -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync: a bit of confusion
xinetd is the daemon. It will spawn rsync processes as connections come to 872 (assuming that's the port you associated with whatever you named that service). This is assuming, of course, that xinetd has read the configuration since you made the change, either by HUP, xinetd bounce, or system bounce. Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] How exactly do I start the daemon? I have it now in xinetd file as, disable = no socket_type = stream wait = no user = root server = /usr/bin/rsync server_args = --daemon log_on_failure += USERID -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync progress indicator?
--progress will show individuals. There is no tracking of total progress, nor any programmatically efficient way of providing such. If you were really concerned, you could --dry-run first and sort of keep track of where you were in the list during the actual run. Tim Conway Unix System Administration Contractor - IBM Global Services - ODCS desk:3039240938 [EMAIL PROTECTED] I've been reading through the man pages for rsync, yet I can't seem to find a way to provide progress indication and/or current download speed for total and/or individual files... Can this be done with rsync? Is it an implemented feature. I thought -v or -vv would do the trick, but it doesn't... Cheers, -- Dag Rune Sneeggen Romolslia 23B 7029 Trondheim Norway dudcore Inc. (http://www.dudcore.net | [EMAIL PROTECTED]) Lovely hosting for elite and not so elite people! Free FTP download mirrors for everyone! (http://mirror.dudcore.net) ___ / I tell ya, dudcore Inc. is good to us \ \ cows! / --- \ ^__^ \ (oo)\___ (__)\ )\/\ ||w | || || -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync crashing Redhat RHEL3.0 server
Congratulations, you've found a weakness in the ethernet driver. Check up2date, as there may be a kernel update for that. You could also maybe plug in a different type of ethernet card. The kernel is explaining that there's a problem at that level, and your rsync log shows what happens when that problem comes up. Don't worry about adding memory (or swap), as you're likely to be using only a few hundred MB when this happens. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] We are using rsync to mirror around 10 of our websites from dedicated server A to dedicated server B runing RHEL 3.0 and rysnc v2.6.3pre1 in a daemon mode. I am using the following crontab script command on dedicated server A: /usr/local/bin/rsync -avz -e "ssh -i /home/users/admin/publicKey-for-server-B" /home/serverA/site3 serverB::home/mirror Dedicated server B (where the mirroring of websites from server A is done) is sometimes crashing with the following error message from the linux kernel: Aug 29 22:47:02 ns2 kernel: eth0: Memory squeeze,deferring packet. Aug 29 22:47:23 ns2 kernel: eth0: NULL pointer encountered in Rx ring, skipping The following is the error message from rsync log in dedicated server B: 2004/08/29 22:45:02 [18367] rsync allowed access on module home from ns.serverA.co.uk 2004/08/29 22:45:02 [18367] rsync to home/mirror from ns.serverA.co.uk 2004/08/29 22:45:51 [18367] site3/logs/ 2004/08/30 00:58:15 [18367] rsync: connection unexpectedly closed (355347 bytes received so far) [receiver] 2004/08/30 00:58:15 [18367] rsync: writefd_unbuffered failed to write 84 bytes: phase "unknown" [generator]: Broken pipe (32) 2004/08/30 00:58:15 [18367] rsync error: error in rsync protocol data stream (code 12) at io.c(909) I've 10 separate crontab rsync scripts for mirroring the 10 websites. I've allowed enough time intervalls so that the each crontab rsync process finishes before the next one starts. Occasionally, however (about once a week) rsync crashes dedicated server B during any one of these 10 crontab processes. I would be very grateful for your help. regards, Daniel. Daniel Berhane bmj.com web administrator The BMJ Publishing Group Limited BMA House, Tavistock Square, London WC1H 9JR http://bmj.com Tel: +44 (0)20 7383 6362 Fax: +44 (0)20 7383 6997 -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Problems of preserving file owership and uid&gid options in rsyncd.conf
move your uid and gid into the module. They are not valid as global options.I hope there's something about auth users and a secrets file somewhere in there as well, as wide-open root to your /var is ill-advised. Ding-- Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I am trying to transfer some files to a remote rsync server. I have to preserve the ownership of these files in the remote server for future possibility of copying them back. The command I am using is: rsync -Cav /var/log/mp3log server2::var/log/mp3log I have tried to add -o and -g options but still can't reach my purpose. I also remember to add a same user name to the remote rsync server. So what's the problem of my command? Another question is: if I don't specify the root uid and gid in rsync.conf, it seems the rsync on the remote server doesn't has enough privilege to create folds, because it is using the uid 99 which is nobody. However, I can't find these same options as mine in the example script files. So what is the proper way to set the uid and gid options?. Any suggestion is appreciated. Thanks! my rsyncd.conf: uid = root gid = root [var] path = /var comment = /var read only = no list = yes Dong-- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Problem related to time-stamp
The environment variable RSYNC_PASSORD and commandline parameter --password-file= are for authenticating against an rsync server. If you're going through ssh, you don't have an rsync question. If you can't "ssh remotehost date" without a password, rsync isn't going to fake up a tty and play expect with ssh for you. Make a passphraseless ssh key, and keep it out of the hands of your enemies. You can even set the authorized_keys file to permit only rsync to be run over that session. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Sorry to disturb u again, but now i have one more very small problem. we need not to supply the password on command line everytime, when we will use rsync for transferring files from one m/c to another,i want to fix login name as well as passord for a particular m/c. I try out the option of seting the environment variable RSYNC_PASSORD and --password-file=FILE . but both of them is not solve my purpose. or may be i m not using them in proper way so kindly gave me ur suggestions regarding the login name and password. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: bash: /usr/local/bin/rsync: Argument list too long
You want everything in wex into wex on the remote. Gotcha. Let's take a simple case. wex contains a b c d e. "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/wex/* [EMAIL PROTECTED]:/mail/spool/imap/user/wex" on the commandline becomes "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/wex/a /mail/spool/imap/user/wex/b /mail/spool/imap/user/wex/c /mail/spool/imap/user/wex/d /mail/spool/imap/user/wex/e [EMAIL PROTECTED]:/mail/spool/imap/user/wex". The parameter ending in "*" is replaced by as many entries as there are in the directory, which can be quite a loth, and is your problem. rsync is perfectly happy to handle freaking enormous numbers of files, but it has to find out about them. Unless there's some valid reason why you want to avoid applying perms,owner,group to the wex directory itself, "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/wex/. [EMAIL PROTECTED]:/mail/spool/imap/user/wex/." will work nicely, or perhaps "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/wex [EMAIL PROTECTED]:/mail/spool/imap/user". These will let rsync build the filelist itself, avoiding problems (and plain old inefficiencies) of argument passing. Speaking of inefficiencies, unless you want to avoid maintaining symlinks devices (not likely to be there anyway) and times, you can improve readability by changing your commandline to "/usr/local/bin/rsync -rsh=/usr/bin/rsh -a --delete /mail/spool/imap/user/wex/. [EMAIL PROTECTED]:/mail/spool/imap/user/wex/." Letting it keep times synced lets it use them to optimize future syncs by not checksumming files that match in name/timestamp/size. "-a" is a lot faster to type than "--owner --group --perms --times --links --recursive --devices" Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I get this error when I try to copy a directory with a lot of files: "bash: /usr/local/bin/rsync: Argument list too long" The exact command is: "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/wex/* [EMAIL PROTECTED]:/mail/spool/imap/user/wex". BUT, if I try tris command it works: "/usr/local/bin/rsync -rsh=/usr/bin/rsh -r --delete --perms --owner --group /mail/spool/imap/user/* [EMAIL PROTECTED]:/mail/spool/imap/user/wex". -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Some files are not getting transferred during the rsync process!!!!
There are many things that can cause that, among them file locking and exclusions. Please provide the specific commandline you are using for the transfer - it's ok to obfuscate the server name and paths. Also identify both platforms. In the meantime, you may wish to run the command more verbosely. Use more "-v"s, up to a maximum of 3, and it'll tell you more of what it's doing or failing to do, and why. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] My first mail to the group. I am using rsync to synchronize two servers. Sometimes, Some of the files are not getting transferred. Is it possible for me to log the files that are missed in this rsync? I tried to get some resources from the net. But I did not find the required info. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Q: Rsync, Batch: How to avoid the "password" ask ?
You're getting there. Save yourself some typing, though, and simplify your commandline until we get auth working. just "rsync [EMAIL PROTECTED]::backup_pa" get the list and we're past the hurdle. Looking at what you're doing, my guess is that you have world read on your secrets file - /etc/rsync.scrt. set it to u+rwx,go-rwx, i.e. 600, and try again... or, if you have no possible security concerns about your logged-in users, leave it whatever you want and add "strict modes = no" to the module so it won't care any more. Note: unless /mnt/hdc1/backup has o+rwx, or nobody ownership, the nobody user isn't going to be able to write. I'd suggest that if you're having only bart2 use it, and he's a real user, have bart2 own it and be the uid. If it's for multiple users files, make it root, and rsync will handle ownerships. DON'T make the rsync auth user name "root" - hell, don't make it anything that has system access. Everything you use runs as the uid anyway. Better yet, give each user his own module, running as himself. If you're safe inside your own lan, ignore my security paranoia. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi Tim, Thanks for the info ... and the congratulations ;) but now when I try to do this in a batch file : SET BSERVER=192.168.0.102 SET RSYNC_PASSWORD=bart2 rsync -av --delete "/cygdrive/c/Doc/backup/Desktop/MesFavories.rar" [EMAIL PROTECTED]::backup_pa I got the following error : @ERROR: auth failed on module backup_pa rsync: connection unexpectedly closed (94 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.2/io.c(342) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync and socket files on HP-UX
No guarantees, but I think sockets are treated similarly to devices by rsync, so instead of the "-a" option, which is equivalent to "-rlptgoD", try "-rlptgo", as in "rsync -rlptgovz -e /usr/bin/ssh [EMAIL PROTECTED]:/tmp /danzas1/dump/testle" Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] i tried to pull files from a linux server to a hp-ux server via rsync -avz -e /usr/bin/ssh [EMAIL PROTECTED]:/tmp /danzas1/dump/testle. It failed for the socket fails. After that i tried to copy files locally on the hp-ux system but same error. Problem seems to be the mknod command on hp-ux that isn't able to build socket files. On Linux this problem doesn't occur it think i don't need the socket files for recovery a system (am i right ?) but i want to get rid of the errors ? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Q: Rsync, Batch: How to avoid the "password" ask ?
on the server side, you seem to have done it correctly. on the client side, you need some changes. "--daemon" is not to tell rsync to connect to an rsyncd. "rsync://" prefex or or "::"seperator do that. "--server is an option passed from an invoking rsync to the one on the other end of an external transport connection - rsh, ssh, maybe even local. I don't know, because it's undocumented, because WE'RE NOT SUPPOSED TO USE IT ON OUR COMMANDLINES. Your second one, the one that asks for a password, is much more sensibly phrased, and should almost work. in its case, you are having nothing to do with the rsync daemon you started on BSERVER. It's an external transport connection. ssh is asking for a password, not rsync. If you'd been connecting to an rsyncd, rsync would have use the password you correctly placed into $RSYNC_PASSWORD in the failed client-side example. SET BSERVER=192.168.0.102 SET RSYNC_PASSWORD=bart2 rsync -av --delete "/cygdrive/c/Doc/backup/Desktop/MesFavories.rar" [EMAIL PROTECTED]::backup_pa/current/bz/favories This tells rsync to connect with its internal transport to the rsyncd running on 192.168.0.102 (on the default port, 873), and place /cygdrive/c/Doc/backup/Desktop/MesFavories.rar in the subdirectory current/bz/favories in the module "backup_pa", i.e., /mnt/hdc1/backup/current/bz/favories. It connects with the rsync username (no relation to any unix username - you made it point to "nobody", as does "root") bart2, using the password "bart2" from the RSYNC_PASSWORD environmental variable. I am surprised it works using a mix of unix path declarations and dos environmental declarations. Since you're already in cygwin, I'd just use pure sh, or if you must run it from the dos environment, i.e. a bat file, I'd specify the windows side path in dos style. Congratulations on getting so far along in the use of a very powerful (and therefore somewhat difficult to master) tool despite the language barrier, and without whining. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi Thanks for the info but either with or without SSH I got errors So my config is : Linux Side (Redhat 9) with rsync 2.6.2 as server Rsync config : rsync --daemon --config=/etc/rsyncd.conf -- /etc/rsyncd.conf: motd file = /etc/rsyncd.motd log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock [backup_pa] path = /mnt/hdc1/backup/ comment = Mon Serveur Rsync uid = nobody gid = nobody read only = no list = yes auth users = root, bart2 secrets file = /etc/rsync.scrt --- -- /etc/rsync.scrt : bart2:bart2 --- Client side : Windows XP rsync 2.6.2 (from cygwin but only needed files not the Unix lookNfeel command prompt) Command file : SET BSERVER=192.168.0.102 RSYNC_PASSWORD=bart2 rsync --server --daemon -ave ssh --delete "/cygdrive/c/Doc/backup/Desktop/MesFavories.rar" [EMAIL PROTECTED]::backup_pa and got this error at client side: rsync: unable to open configuration file "rsyncd.conf": No such file or directory rsync error: syntax or usage error (code 1) at /home/lapo/packaging/tmp/rsync-2.6.2/client server.c(498) The next command works fine at client side even if the linux rsync daemon is NOT launched. rsync -ave ssh --delete "/cygdrive/c/Doc/backup/Desktop/MesFavories.rar" [EMAIL PROTECTED]:/home/bart2/backup/current/bz/favories But it allways asks for a password ! Thanks in advance for your help Regards, Bart. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync + ssh
Well, whaddya know? I've known of people running a rsyncd binding on localhost and tunneling, when they were afraid of getting sniffed. I assume this is just an automation of that process? I guess it's time to reload my mental copy of the man pages. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] According to the manpage and other docs, it also possible to use ssh to connect to a daemon using "::". However, you must use the --rsh="ssh ..." option instead of the -e. "o for copying from the local machine to a remote machine using a remote shell program as the transport, using rsync server on the remote machine. This is invoked when the destination path contains a :: separator and the --rsh=COMMAND option is also provided. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync + ssh
That's an ssh problem, not rsync. try "ssh xx.yy.zz". That said, ssh is irrelevant to what you're trying to do. you are going to a module on an rsync daemon, not a directory through a remote shell transport connection. :: is an rsync-protocol TCP connection to port 873, not an ssh-protocol TCP connection to port 22. Pick one. Either "rsync -aurvlpogt kk.txt XXX.YYY.ZZZ::pruebas1", or " rsync -aurvlpogt -e ssh kk.txt XXX.YYY.ZZZ:/path/to/pruebas1". For the latter, you'll have to get ssh working between the two systems. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hello gurus I attemp to make a backup between two hosts, using rsync, and result is ok. But and I agree "-e ssh" to sentence, host backup says rsync -aurvlpogt -e ssh kk.txt XXX.YYY.ZZZ::pruebas1 @ERROR: access denied to pruebas1 from unknown (0.0.0.0) rsync: connection unexpectedly closed (78 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(342) Any idea ?? Thanks... -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: IO error encountered - skipping file deletion
That is a safety mechanism. If there was an error, the lists may be in error, and a deletion may take out a whole lot more than you want. Picture getting permissions on a large section of the source directory messed up so the rsync process can't see into them. The send list now does not include everything down there, but it can be seen, and deleted, on the destination. Generally, users would prefer to rectify the problems on the source end and re-run the sync, rather than losing large quantities on the destination end. This safety mechanism can be overridden, by the "--ignore-errors" flag. If you're taking the default --delete behaviour, wherein it deletes before the transfer (can be overridden by "--delete-after"), these errors are in the filelist generation stage, not problems of insufficient space, or dropped connection. If you do, in fact, have a place you can't read on the destination or source, and it should be that way, exclude it. That way, it won't hit it and error, so you can both run your sync with deletes, and still have it try not to ruin your life over a little problem. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] hello, "receiving file list ... 96 files to consider IO error encountered - skipping file deletion wrote 101 bytes read 2047 bytes 1432.00 bytes/sec total size is 107673960 speedup is 50127.54 rsync error: partial transfer (code 23) at main.c(926)" Why can't delete files? Thank you! -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsycnc copies all files
I'm glad. I remember a brief debate about it, and I thought that my side lost. It always seemed to me that it should go on a case-by-case basis, rather than assuming that NFS over gigabit was slower than ssh over dialup. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I haven't heard that discussed. I think we'll just leave it as it is, where it only defaults to --whole-file if it's doing a local transfer. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsycnc copies all files
Use "-a" unless there's a reason you need to retain different permissions, owners, etc.. "-c" is pretty much for those situations wherein you end up with identical times and size but different content, and you need rsync to look inside even though it appears unnecessary. As you have seen, slower than heck. I don't know the nature of your filesystems, but I have a guess... on at least one end is a network filesystem - NFS, SMB, NCP, AFS, something like that. rsync has the "-W", or "--whole-file" option, which tells it that there's no point in trying to read the file for changes - if it needs transferring, just send the whole thing, because the LAN overhead will waste the savings in WAN. Last I heard, -W was going to be forced if either end was a network filesystem. Wayne: did that ever come to pass? Another thing: I haven't seen your actual commandlines and results, just descriptions. Here's a set of sample runs. Note the difference between "total size" and "wrote", and the fact that 29 or so bytes + 73 bytes nop overhead doesn't come close to 470, or 6806. +++ [EMAIL PROTECTED] /home/cnwt99/work>sh doit mkdir source ssh disaster mkdir destination dd if=/usr/local/admin/lib/locatedb of=source/Kfile bs=1024 count=1 1+0 records in 1+0 records out rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile disaster:destination building file list ... done Kfile wrote 1137 bytes read 36 bytes 782.00 bytes/sec total size is 1024 speedup is 0.87 rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile disaster:destination building file list ... done wrote 73 bytes read 20 bytes 62.00 bytes/sec total size is 1024 speedup is 11.10 date >>source/Kfile rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile disaster:destination building file list ... done Kfile wrote 470 bytes read 48 bytes 345.33 bytes/sec total size is 1053 speedup is 2.30 rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile disaster:destination building file list ... done wrote 73 bytes read 20 bytes 62.00 bytes/sec total size is 1053 speedup is 11.32 dd if=/usr/local/admin/lib/locatedb of=source/Mfile bs=1024 count=1024 1024+0 records in 1024+0 records out rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile disaster:destination building file list ... done Mfile wrote 1048813 bytes read 36 bytes 699232.67 bytes/sec total size is 1048576 speedup is 1.00 rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile disaster:destination building file list ... done wrote 73 bytes read 20 bytes 62.00 bytes/sec total size is 1048576 speedup is 11275.10 date >>source/Mfile rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile disaster:destination building file list ... done Mfile wrote 6806 bytes read 9024 bytes 6332.00 bytes/sec total size is 1048605 speedup is 66.24 rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile disaster:destination building file list ... done wrote 73 bytes read 20 bytes 62.00 bytes/sec total size is 1048605 speedup is 11275.32 [EMAIL PROTECTED] /home/cnwt99/work>date Thu Jun 17 16:11:23 MDT 2004 [EMAIL PROTECTED] /home/cnwt99/work>date |wc -c 29 [EMAIL PROTECTED] /home/cnwt99/work> +++ Note also that the delta transfer doesn't happen on local transfers. Since to do so would be to build a new copy of the file at the destination, reading from both the source and destination files (once in entirety for the checksums, again partially for the pieces kept and sent), then deleting the destination, then renaming the newly created file to the destination, it just does it in one read/write/unlink/rename... essentially like the "-W" option. Good luck. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Wayne Davison wrote: >On Wed, Jun 16, 2004 at 08:26:33PM +0100, Gareth wrote: > > >>I am making an extraordinary claim: rysnc seems to copy all my files, >>not just ones that have changed or new files. >> >> > >Use either -t (preferred) or -c (slower). See also -a. > >..wayne.. > > Using -a, -t or -c (vvv slow!) does resolve the problem, but now I notice that appends to text files results in the whole file being transferred (as recorded by 'Total bytes written') rather than just the 9 characters I append to the file. Surely this isn't the desired outcome? Any information much appreciated. Gareth -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Problem in using rsync
Classic. I used to see that. In mine, I finally had to give up, and wrote another tool... not rsync's fault. I would get timeouts during file list builds. As I recall, there's an internally-defined "SELECT_TIMEOUT", that, at least back then, remained at 60 seconds, regardless of the commandline timeout. Now that you've boosted the speed, your big runs are finishing their filelist builds and taking off on hard I/O usage, slowing the filelist build enough to exceed the SELECT_TIMEOUT. Once the list is built, it's more robust. Try running the other rsyncs niced. They'll still burn like crazy, but the increased CPU demand of the big list run during its list build may hold them down enough to let it finish building. Also, start the biglist run first, so its CPU will be busy. If it's the one sharing, it'll be slowed. 10 seconds should suffice, if I'm right. If that doesn't do it, give it a bigger head-start, long enough to start the transfer. The easiest way to determine that timeis to run it once to completion, then run it again on unchanged data, and note the time for that operation. That's much head-start it needs to be in transfer before the big dogs choke it. Wayne'll probably correct my errors. That SELECT_TIMEOUT thing has probably changed by now. It's been a couple of years since I read and mentally traced the whole tree. But, nicing is still a good bet, as is the head-start. Your timeout is already pretty substantial. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Anh Truong <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 06/17/2004 08:11 AM To [EMAIL PROTECTED] (Receipt Notification Requested) cc Subject Problem in using rsync Hi I use rsync to perform backup on disk on a SunFire 880 with Solaris 8. For performance issues, we launch simultaneously 5 rsyncs on 5 different fliesystems and about 150-200 "cp -p" commands on as many database files. We have been using the same scripts for about 2 months, without problems. The backup is performed on the same server (from filesystem to filesystem on the same server). Last weekend, we replaced the 4X750MHz by 8X1200MHz CPU's and upgraded from 8 to 16 MB of RAM. Since then, we had 2 errors out of 3 backup runs. The error is always on the same filesystem, which is not the largest one but the one that has the more files and directories (400 000 files as opposed to 600 for others). The error message we have is: --- io timeout after 600 seconds - exiting rsync error: timeout in data send/receive (code 30) at io.c(143) rsync: writefd_unbuffered failed to write 69 bytes: phase "unknown": Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(836) The command used is: OPTS="--delete --timeout=600 --exclude dbf/ --rsh=rsh" /usr/bin/rsync -vRa $OPTS ORIGIN_DIRECT DESTINATION_DIRECT In the documentation, it is said that this kind of error might be related to either: Disk full = This is not the case Remote rsync is not found = It is on the same server, so it is found remote-shell setup isn't working right or isn't "clean" = I tried the suggested testing and there is no problem. Moreover, it worked before... I saw also that the rsync process might have been starving for CPU or memory, In our case I do not think it might be the case. Can you help me on this??? Thanks in advance -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Truly awful rsync docs - Re: real Newbie query sorry!
It's an open project, and there's lots of work to be done besides programming. I'm sure a nice manual for the technically-challenged would be welcomed. Most users of rsync posess great technical competence, and aside from unavoidable language issues(we didn't all grow up in OZ, UK, NZ, or US), they find their way through to the parts they need, and get extra guidance from this list for the more arcane features and for the normal mental blocks (It's amazing how many people unconsciously edit "::" to ":" when they read how to access an rsyncd). PHB-types will probably require a GUI, with audio prompts and lots of pretty colors, and still not know what to do with it. People do write such wrappers. Show us what you write. It's of no interest to me for my own use, but I will set aside the time to read it for the project. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: trigger command on successful upload?
If you're rsyncing over an external transport, I can't imagine what the issue is in running a postprocessing command over the same transport. But, if what you're looking for is a way to do this over rsync internal transport, and want to get rid of the remote shell, perhaps for security reasons, leave a daemon, or perhaps a cron-driven script, to watch for the appearance of a trigger file. If the rsync transfer finishes correctly, you send up the trigger file, which is harvested by the receiver, which performs the desired post-transfer processing. start pseudoscript:+++ [ -f "$triggerfile" ] || exit 0 rm $triggerfile perform whatever it is you wanted end pseudoscript:+++ Cronjob: * * * * * /path/to/the/script Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I would like to be able to trigger a script for #3 instead of having to ssh over (to make the frontend server a bit more autonomous, and simplify the process). Does this seem reasonable, or am I just making things more complex for myself? :) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Netiquette: Out of Office AutoReply and mail reflectors
If you must use an out-of-office feature in your email, please exclude any mailing lists you're subscribed to. I deleted the message that I was forwarding in this, but I'm sure we all know what I'm talking about. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: installation problems
I would suggest installing a compiler. GCC should be on your installation set. If not, it's a simple matter to download it. If you already have one, check your $PATH. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hallo! I am trying to install rsync on fedora core2. When I run ./configure I get the following error message: [EMAIL PROTECTED] rsync-2.6.2]# ./configure configure: Configuring rsync 2.6.2 checking build system type... i686-pc-linux-gnuoldld checking host system type... i686-pc-linux-gnuoldld checking target system type... i686-pc-linux-gnuoldld checking for gcc... no checking for cc... no checking for cc... no checking for cl... no configure: error: no acceptable C compiler found in $PATH See `config.log' for more details. How do I solve this problem? Thanks! -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [Fwd: Re: rsync server complaining about vanishing files while they are not.]
Hans: Unless your server is in a completely trusted network, I'd suggest you put down some include/exclude rules for that module, because people can poke about anywhere they want. [module] path = /only/directory/you/want/to/give/them use chroot = no user does rsync server::module/../../../../../../../etc/security/passwd . Permissions (don't put "uid = 0" in the rsyncd.conf) can prevent that one, but rsync -a server::module/../../../../../../.. . is probably something you don't want. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Yep, I have set "use chroot = no" for the module and it works. The symlinks I have are created automatically by a tool and are absolute. But since "use chroot = no" handles well absolute links from root/, the transfer works A1. Thanks for your input, Wayne. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync 2.6.2 doesn't work with GNU inetutils rsh
The library I referred to was getopt. It just seems to be trying to be a bit too clever... kind of like the automatic tuning radios in Hitch Hiker's Guide to the Galaxy. I didn't know about the existing oddity in rsh. Thanks for the education. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hmm. To be fair, rsh has always been an outlier - it's allowed flags to follow the hostname ever since I can remember. And there are other fine examples of tradition like tar and dd which are hardly exemplary. If anything, I'd feel things are slowly getting better. But for interest, which new library do you refer to above? > You will have to write an rsh wrapper. Hmm. That was actually what I did yesterday. It's not the right solution to expect every end user to do that though. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync 2.6.2 doesn't work with GNU inetutils rsh
A few weeks ago, I corrected a guys commandline, wherein a flag came after a directory specification, and was informed that new libraries rearrange ARGV. I was somewhat abashed, not having known that, but also thought that was a really stupid idea. Now we have to do kludgy workarounds in order to accomodate users who are too stupid to understand a simple syntax. You will have to write an rsh wrapper. This will probably do: #!/bin/sh #rsh4rsync - wrapper to insert "--" host=$1 shift if [ "$1" = "-l" ] then rshargs="$1 $2" shift;shift fi exec rsh $host $rshargs -- $@ I don't think rsync ever adds more than a -l user (taken from [EMAIL PROTECTED]:path) to the transport commandline. No guarantees. I wrote that in here, and haven't tested it. Now, if sh's argument parser starts fscking with your commandline, you're hosed. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] This is the cygwin build of rsync, with the standard cygwin rsh (which is a fairly old GNU inetutils 1.3.2). ~=> rsync --rsh=rsh -vv bibble: opening connection using rsh bibble rsync --server --sender -vvr . rsh: unknown option -- server As it helpfully explains, rsh is grabbing all the arguments intended for the remote rsync command. With GNU rsh, it seems necessary to add a '--' to provide a limit: rsh bibble ls -l : error from rsh rsh bubble -- ls -l : works fine -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hangs in cron (not SSH-problem)
I would suggest that you put those commandlines into scripts, and redirect stdout, stderr, AND stdin - "rsync -av --delete /mnt/web1 /mass/kuurne/day logfile 2>&1", for instance, or if you're wanting it just mailed like cron will do, "rsync -av --delete /mnt/web1 /mass/kuurne/day &1 |mail $USER"... though I'd do a file, send(see next paragraph). At one time, on some systems, rsync run from cron detected stdin as a socket and behaved as if it were called from inetd as an rsyncd. Plainly, your problem isn't as simple as that, as all three necessary processes for a normal local sync are coming up, but eliminating unknowns in the filehandles it gets is a good idea. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] >>When used this command in cron >> >>00 01 * * * rsync -av --delete /mnt/web1 /mass/kuurne/day >>00 02 * * * rsync -av --delete /mnt/web2 /mass/kuurne/day >>etc.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: include directory and all files under
Oh, of course. I just meant I've never needed it, and the original question was raising an unnecessary application of it, not that the function is useless. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] the main use i've found for them is if you have tons and tons of -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: (no subject)
Run it this way:
/usr/local/bin/rsync -aHnuvvv serverX:/ / --exclude-from=/rsync.exclude
--rsync-path=/usr/local/bin/rsync --ignore-existing 2>&1 |tee
/tmp/rsync.debug
The extra verbosity will show you what transport you're using, and other
problems. It could be that you're sshing in, and the authorized_keys file
on serverX has command restrictions, for instance, so it just throws you
out as soon as rsync invokes the remote. Combining stdout and stderr lets
you see how they fit together. tee just lets you watch as it goes.
Good luck.
Tim Conway
Unix System Administration
Contractor - IBM Global Services
desk:3032734776
[EMAIL PROTECTED]
>From the local machine the following command is executed:
/usr/local/bin/rsync -aHnuv serverX:/ / --exclude-from=/rsync.exclude
--rsync-path=/usr/local/bin/rsync --ignore-existing >
/var/tmp/rsync.stdout 2> /var/tmp/rsync.stderr
I have never used the rsync command. The above command was used by a
former sysadmin to "synchronize" two servers. However when I ran the
command it did not produce any "dry -run" output ("n" option). A check of
the /var/tmp/rsync.stderr shows the following output ws produced: "rsync:
connection unexpectedly closed (0 bytes read so far); rsync error: error
in rsync protocl data stream (code 12) at io.c(165)."
Based on the above syntax, what was included/excluded that made this not
work, and what is the correct syntax to "sync" the local machine to the
remote?
In advance, thanks for the help.
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Unexplained error (code 24)
/home/cnwt99/rsync-2.6.2>grep ' 24 ' errcode.h #define RERR_VANISHED 24 /* file(s) vanished on sender side */ [EMAIL PROTECTED] /home/cnwt99/rsync-2.6.2> http://www.m-w.com/cgi-bin/dictionary?book=Dictionary&va=vanished&x=0&y=0 Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Graham Leggett <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 06/01/2004 09:26 AM To [EMAIL PROTECTED] cc Subject Unexplained error (code 24) Hi all, While trying to mirror a filesystem from one machine to another (for backup purposes) I get the following error: [EMAIL PROTECTED] root]# /usr/bin/rsync -qavxzC --delete chandler:/var/ /bigdisk/backup/chandler/dev-md5-var/ [EMAIL PROTECTED]'s password: rsync error: unexplained error (code 24) at main.c(1045) Does this mean anything to anybody? Regards, Graham -- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: include directory and all files under
rsync -a&otheroptions rsync://carroll.cac.psu.edu/openbsd/snapshots/i386 rsync://carroll.cac.psu.edu/openbsd/snapshots/ports.tar.gz /some/destination/ /some/destination will look liks snapshots but with only i386 and ports.tar.gz in it. You did know that you can specify multiple sources, right? Oh, you could also get it in two steps, for instance, if you wanted to put the contents of i386 up with ports.tar.gz If you insist on using include/excludes, it's a lot tougher. + /i386 + /ports.tar.gz - /* If you exclude '*', you'll also have to explicitely individually include every item under i386... probably not part of your plans. I personally have never used include/excludes, but I remember reading the man page a few years ago, and though it was very well written, I've never had a scenario where they were appropriate. From your specification, I don't think you do either. Simplify. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Harry Putnam <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 06/01/2004 09:33 AM To [EMAIL PROTECTED] cc Subject include directory and all files under Sorry, but it seems everytime I setup a different rsync operation (client only) I end up here unraveling the include/exclude stuff I need. The documentation is quite good but I guess the subject is just quite a bit to chew. To cut to the chase: I want to sync up my own snapshot repository of openbsd. The basic install files. They reside at: rsync://carroll.cac.psu.edu/openbsd/snapshots/i386/* However, there is also a `ports' source file that I want to include that resides at: rsync://carroll.cac.psu.edu/openbsd/snapshots/ports.tar.gz One level up from the other stuff. At that level, there are lots of different architectures etc I don't need so want to exclude. I'm trying this approach rsync [...] (various flags skipped) --include-from="./include" --exclude="*" rsync://carroll.cac.psu.edu/openbsd/snapshots/ /some/destination/ ./include looks like: cat include: ports.tar.gz i386 That gets the ports.tar.gz and and empty directory named i386 Trying: ports.tar.gz i386/* Doesn't get i386 or anything under it: Ditto for this: ports.tar.gz rsync://carroll.cac.psu.edu/openbsd/snapshots/i386/* or ports.tar.gz */i386/* So how is something like this done? And what do I need to know to be able to figure it out next time? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Keeping Multiple Rsyncs Separate
They are kept seperate. If one tried to use another's chunk of memory, it'd be a segfault. To permit such interference (and potential coordination) would require using shared memory, and nobody wants to deal with that. If you are, in fact, having one process modify memory belonging to another, that is a defect in your VMM. If you have two rsyncs to the same location, results are indeterminate, since it's a race condition. The above fact explains why they do not and can not coordinate. Only a fool permits two uncoordinated processes to modify the same set of data at the same time. Run one, then the other. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I have noticed that if you run two rsyncs at once, they get confused and copy the files from one the wrong rsync thread. Apparently this is because of the âBuild Listâ that is made in ram. Two build lists stepping on each other. Does anyone know how to change the source so that the each build list in ram is kept separate? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Bidirectional speed question
I agree with Paul. It's almost certainly hour WAN link. My own at home often gives sustained downloads in excess of 2Mbps, This seems to be throttled by the cube of the difference between upload speed and 16kbps. When I get up to 10kbps up, it's still useable. At 12, it's like a 21,400 dialup, at 14, it's like amateur packet radio, and at 15-15.9kbps upload, all downloads cease... even name resolution fails. Sprint Broadband Direct is an extreme example, but most home broadband has similar linkage between the pipes. You are likely to actually improve your overall performance by throttling the rsync with the --bwlimit= option. Turn the upstream one's speed down to about 70% of its maximum alone, and it should leave enough for the downstream to do well. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] 2. Line speed is around 100 kbps but halves when transferring files in both directions. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: error in rsync protocol data stream
It appears that the command you show is not a complete copy/paste job. When you changed "backup.amlaw.com" or whatever it was to "backup.domain.com", you also removed one of the colons between that and "jspfsp", as the only time rsync does a chroot is as a server. A chroot failure is almost invariably an erroneous "path =" line in the rsyncd.conf, and since it works otherwise, we know your OS can do chroot. Check the directory named under the "[jspfsp]" entry in your rsyncd.conf. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I am doing this to rsync a file: rsync -avz /usr/local/websphere/appserver/hosts/default_host/jsp/servers/includes/file1 .txt backup.domain.com:jspfsp >> /var/log/rsync_backup.log I get these error: @ERROR: chroot failed rsync: connection unexpectedly closed (34 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) I am rsyncing several folders between these two servers with no problem. But now I keep getting this rsyncing a single script. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: A question about rsync
That is it. The destination file is unaffected until rsync completes its replacement, then the directory entry is repointed at the new file and the reference to the old inode freed. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi: I really want to know how rsync works. Once it synchronize a file. Does rscync first create a temporary in the remote machine first and then rename it? Or it direct write the difference into the dest-file? Could you please tell me what will happen to the dest-file when a rsync process interrupted by some problems(network problem etc ...)? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: cwrsync strange path in error message
That's where main.c was when the binary was compiled. It's telling you that if you took the same source code and looked at line 383 of main.c, you would see the line that generated that error 23. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] 2004/05/11 [93] rsync error: some files could not be transferred (code 23) at /home/lapo/packaging/tmp/rsync-2.5.7/main.c(383) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync output -vv differs with dry-run option
The dry run was successful, and transferred 0 bytes. The dry run is for a quick check, and will show what objects, if any, would be transferred in a real run, not exactly how many bytes would be transferred. Perhaps a --write-batch in a dry run could create the batch files, and you could just wc them. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I'm trying to figure out if a file has changed since the last rsync call. I use the following command line: rsync -cvv /mnt/xxx/vol1/dbase/100/kunden.dbf /mnt/label | grep "^total: " | sed -e 's/.* data=//' This gives a 0 if the file is unchanged and the file size if the file has changed. Adding the "dry-run" option "n" to the command line always gives a 0. I wonder if this is a expected behaviour? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: progress & redirects
How about not using --progress when it's running from cron? ... unless you're tailing the logfile, which should also work fine as-is. If you need to process the logs otherwise later, feed it to "sed 's/^H.*^H///'" to dump the display crap... or is it "^M" instead of "^H"? Those aren't literals. Produce them by doing a control-V followed by control-H or control-M as needed. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi. I use rsync to suck down a large amount of data every night using a cron job that logs to a file. If you run rsync --progress and redirect to a log file you end up with the progress for each file piled up onto a single line. \r is generally ignored by editors and viewers. That leads to my question... Would it be possible to have rsync output log-friendly progress if output is redirected? For an example of how this can work, check out wget. When run in a shell, it outputs a beautiful, dynamic progress bar. When output is redirected it outputs periods instead. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync and Perl programming
escape your @. I don't remember the details, but I know it got chewed up
by something in a past application.
Tim Conway
Unix System Administration
Contractor - IBM Global Services
desk:3032734776
[EMAIL PROTECTED]
Hi everybody -
I'm trying to write a Perl wrapper for some rsync tasks that need doing.
Problem is, there's some sort of odd interaction going on between Perl
and the daemon mode communication for the rsync client, and I'm at my
wit's end in trying to figure it out.
Here's the Perl script:
#
#
#!/usr/bin/perl
$rsync_cmd = "/usr/local/bin/rsync -a --progress
--password-file=/usr/local/etc/rsyncd.passwd
[EMAIL PROTECTED]::samba/rsynctestfiles /usr/share/smbshare/rsynctestfiles";
$pid = open(PH, "$rsync_cmd 2>&1 |"); # with an openpipe
while () {# plus a read
print $_;
}
print "finished\n";
#
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (code 12) at io.c(189)
Standard newbie issue. rsh host command (or ssh host command in your case) does so in a fairly stripped environment, for instance, a small $PATH, which is your issue (unless rsync isn't even installed on smmk39). rsync has aneasy workaround for it. add " --rsync-path=/path/on/the/remote/host/to/rsync " commonly " --rsync-path=/usr/local/bin/rsync ". You can also link it into someplace in your path, if you're the admin, and are going to be having a lot of users doing their own rsync commandlines. The stuff you're doing with services and inetd.conf are irrelevant, unless you're going to run an inetd-managed rsyncd, which is NOT what you are doing or accessing in the commandline given. Also your inet hupping is a bit scary. If you have anything that has the word "inetd" in the ps line, it's going to get a HUP, and many things react to a HUP like to a TERM. I'd suggest a pattern like " /usr/sbin/in[eE]td -s$", as you're on Solaris 8. To call that rsync server, forget the "-e ssh". have this in your /etc/rsyncd.conf: ++ [home] path = /home ++ Or tighter yet: ++ [homeforbob] path = /home/a078479/bob ++ rsync -va /export/home/a078479/bob smmk39::homeforbob I see, having read to the bottom of your message, that you did the linking thing. That's cool, and well-documented. Glad you figured it out. It's not an rsync issue, but an environmental one. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Rsync Issue Solaris8 When performing a simple rsync between servers I was getting the following error: root:#> rsync -e ssh -va /export/home/a078479/bob smmk39:/export/home/a078479/ ksh: rsync: not found rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) root:#> -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Simultaneous rsyncs?
The last version to finish and be renamed over the existing target file would be the one you end up with. No mixing, and no error returned, either to the winner or the loser, just a perfect copy of the last version sent, just as if you had two people editing the same file in a single directory... the last one to save wins. If you're going to have multiple modifiers per file, you must resolve control. You're right, you do "need some sort of queuing or interlock mechanism", not just to prevent simultaneous rsyncs, but simultaneous edits. That's why we have things like CVS, RCS, SCCS, PVCS, BitKeeper, etc.. By the way - rsync gets along VERY well with CVS. If this directory is already under version control(even just good coordination among personnel), and it's going out for read-only use, you're sailing smooth. When one rsync finishes and moves its temporary file over, it unlinks the old file and puts the new one in its place, so even if a subsequent rsync is still using the destination file for a template, it won't matter, because it'll still have the inode open, and will work from the now nameless original file, and release it when it finishes, then delete the "new" original, and put its version in place. On the other hand, I can't say what the consequences would be of modifying the source file in-place during a send. I expect rsync notes changed mtime and restarts. Wayne? Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] What would happen if two people ran rsync on the same set of source files to the same destination machine? Do we need some sort of queuing or interlock mechanism to prevent simultaneous rsyncs? More problematical is when two or more distributions overlap. What would happen in that case? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync Error..
I gave the example, and understood that you'd used it. If instead of " rsync -e rsh --rsync-path=/usr/local/bin/rsync 66.123.34.123:/etc/services ", you used "rsync -e rsh 66.123.34.123:/etc/services" and it worked, then you don't need the --rsync-path= option. If however, the second example above fails, and the first one works, that's what you need, and it contains the exact correct syntax for your use. Someone has done some really good work writing and formatting the man pages. These give more detail and explain much more precisely than I can. You might want to take a quick glance. - for instance, see below: SunOS 5.8 Last change: 26 Jan 2003 15 User Commandsrsync(1) --rsync-path=PATH Use this to specify the path to the copy of rsync on the remote machine. Useful when it's not in your path. Note that this is the full path to the binary, not just the directory that the binary is in. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Naveen Babu <[EMAIL PROTECTED]> 04/26/2004 11:41 AM To Tim Conway/Denver/Contr/[EMAIL PROTECTED] cc [EMAIL PROTECTED] Subject Re: Rsync Error.. Tim Conway <[EMAIL PROTECTED]> wrote: assuming you replaced "hostwithrsync" with "66.123.34.123", you're in. Yes, i replaced it with the ip address. I have a question about rsync-path option. The command I am giving at the source machine is: rsync -aznrbe rsh /sourcepath 66.123.34.123:/destinationpath Could you please tell me how to use "rsync-path" option for the above command. Thanks, Naveen. It's fine that your /etc/services is a symlink... the important thing is that you can see it with rsync over rsh. Use the rsync-path option, and enjoy the tool. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Naveen Babu 04/26/2004 09:05 AM To Tim Conway/Denver/Contr/[EMAIL PROTECTED] cc [EMAIL PROTECTED] Subject Re: Rsync Error.. Hello Tim, Sorry for the late response. I was out of town for weekend. Ok, first I tried rsync -e rsh --rsync-path=/usr/local/bin/rsync hostwithrsync:/etc/services I got the following the output: -rw-r--r-- 73490 2003/10/27 09.26.50 services ++ lrwxr-xr-x 15 2003/08/19 08:13:58 services + The file persmiss! ions are different from what you mentioned. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync Error..
assuming you replaced "hostwithrsync" with "66.123.34.123", you're in. It's fine that your /etc/services is a symlink... the important thing is that you can see it with rsync over rsh. Use the rsync-path option, and enjoy the tool. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Naveen Babu <[EMAIL PROTECTED]> 04/26/2004 09:05 AM To Tim Conway/Denver/Contr/[EMAIL PROTECTED] cc [EMAIL PROTECTED] Subject Re: Rsync Error.. Hello Tim, Sorry for the late response. I was out of town for weekend. Ok, first I tried rsync -e rsh --rsync-path=/usr/local/bin/rsync hostwithrsync:/etc/services I got the following the output: -rw-r--r-- 73490 2003/10/27 09.26.50 services ++ lrwxr-xr-x 15 2003/08/19 08:13:58 services + The file persmissions are different from what you mentioned. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync Error..
Jim: first - glad to see you're able to reach the group again. Second: In later messages, he'd found and fixed true rsh - to where "rsh remotehost which rsync" returned "/usr/local/bin/rsync", and still got the timeout on actual rsyncs. I sent an enormous number of possible outputs he could get and what they'd mean, and haven't heard back. I think that might have been late Friday. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Tim already answered your question for you. Your remote system is not answering on port 514, only on port 513. See your OS's rsh man page for further details - this isn't really an rsync question, it's an rsh question. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync Error..
Ok, I'd expected it to go once you could rsh host which rsync. Since you're not getting a + [EMAIL PROTECTED] /usr/local/admin/bin>rsync -e rsh --rsync-path=/usr/local/bin/rsync hostwithoutrsync:/etc/services sh: /usr/local/bin/rsync: not found. rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) [EMAIL PROTECTED] /usr/local/admin/bin> + I don't think it's the remote host binary, but just in case, try a plain rsync -e rsh --rsync-path=/usr/local/bin/rsync 66.123.34.123:/etc/services I remember once upon a time, rsync choked on verbosity, which is why I left out the "v"s. If that command fails, try it with "vvv". It doesn't tell much more than with none at all, but I'd like to see it. + [EMAIL PROTECTED] /usr/local/admin/bin>rsync -vvv -e rsh --rsync-path=/usr/local/bin/rsync hostwithoutrsync:/etc/services opening connection using rsh hostwithoutrsync /usr/local/bin/rsync --server --sender -vvvr . /etc/services sh: /usr/local/bin/rsync: not found. rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12) [EMAIL PROTECTED] /usr/local/admin/bin> + I'd expect more something like this + [EMAIL PROTECTED] /usr/local/admin/bin>rsync -e rsh --rsync-path=/usr/local/bin/rsync hostwithrsync:/etc/services lrwxr-xr-x 15 2003/08/19 08:13:58 services [EMAIL PROTECTED] /usr/local/admin/bin> + Or like this, with three "v"s. + [EMAIL PROTECTED] /usr/local/admin/bin>rsync -vvv -e rsh --rsync-path=/usr/local/bin/rsync hostwithrsync:/etc/services opening connection using rsh hostwithrsync /usr/local/bin/rsync --server --sender -vvvr . /etc/services add_exclude(/*/*,exclude) add_exclude(/*/*,exclude) server_sender starting pid=6090 make_file(1,services) expand file_list to 4000 bytes, did move recv_file_name(services) received 1 names lrwxr-xr-x 15 2003/08/19 08:13:58 services recv_file_list done get_local_name count=1 generator starting pid=5968 count=1 delta transmission enabled generate_files phase=1 recv_files(1) starting send_file_list done send_files starting recv_files phase=1 generate_files phase=2 send_files phase=1 send files finished total: matches=0 tag_hits=0 false_alarms=0 data=0 recv_files finished wrote 24 bytes read 331 bytes 236.67 bytes/sec total size is 15 speedup is 0.40 _exit_cleanup(code=0, file=main.c, line=1045): about to call exit(0) [EMAIL PROTECTED] /usr/local/admin/bin> + Instead, you're getting this: + [EMAIL PROTECTED] /usr/local/admin/bin>rsync -e rsh --rsync-path=/usr/local/bin/rsync hostwithoutrsh:/etc/services hostwithoutrsh: Connection timed out rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) [EMAIL PROTECTED] /usr/local/admin/bin> [EMAIL PROTECTED] /usr/local/admin/bin>rsync -vvv -e rsh --rsync-path=/usr/local/bin/rsync hostwithoutrsh:/etc/services opening connection using rsh hostwithoutrsh /usr/local/bin/rsync --server --sender -vvvr . /etc/services hostwithoutrsh: Connection timed out rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12) [EMAIL PROTECTED] /usr/local/admin/bin> + If this one fails, rsh 66.123.34.123 /usr/local/bin/rsync / Maybe your timeout is an rsync timeout instead of an rsh timeout. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hello, I uncommented shell in inetd.conf on the destination system. The command " rsh 66.123.34.123 which rsync" is working fine now . I get the output /usr/local/bin/rsync. But the command: rsync -avvznrbe rsh /sourcepath 66.123.34.123:/destinationpath Still does not work. I get the same error as i used to get before. opening connection using rsh 66.123.34.123 rsync --server -vvbnlogDtprz . /destinationpath 66.123.34.123: operation timed out rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(18
Re: Rsync Error..
Ah! Now I've got you. rsh without parameters does an rlogin, which uses the service commonly called "login", on port 513. With parameters, it does a plain rsh, using the service commonly called "shell", on port 514. Here's a system with both holes open (deep inside a well-protected intranet, host and usernames deleted to protect the innocent). egrep "^shell|^login" /etc/services /etc/inetd.conf /etc/services:login 513/tcp /etc/services:shell 514/tcp cmd # no passwords used /etc/inetd.conf:shell stream tcp nowait root/usr/sbin/in.rshd in.rshd /etc/inetd.conf:login stream tcpnowait root /usr/sbin/in.rlogind in.rlogind You probably have the shell line in inetd.conf commented out. open it, HUP inetd, and you're probably in business. No sane security guys could complain about shell if login is permitted. I'm pretty sure that's your issue. It's also possible, of course, that you shell service is on an alternate port, or commented out of services. You'll see it, whatever it is. All ssh sessions go through the sshd service, on port 22, and most systems that have one or the other of login and shell enabled have both. Yours is just an unusual case. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hello, when i gave "rsh 66.123.34.123 which rsync" at the source system i got the following error 66.123.34.123: Connection refused "rsh 66.123.34.123" works perfectly fine. What could be the problem..? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync Error..
rsh 66.123.34.123 which rsync I expect you'll get something like "no rsync in /usr/bin /usr/ccs/bin /usr/bin/X11 /usr/contrib/bin /usr/local/bin ." Some systems don't report the remote shell connection open until the called remote program comes up, so one missing from the path can look like a network timeout. I don't remember if that's true of any FreeBSD. If this is the case, use --rsync-path=/wherever/it/is/on/66.123.34.123 Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] opening connection using rsh 66.123.34.123 rsync --server -vvbnlogDtprz . /destinationpath 66.123.34.123: operation timed out rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Error?
OK, I'm stumped. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] 1 2 * * * rsync --daemon 0 8 * * * killall rsync -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Error?
Can we see the crontab line? For some reason, you're repeating the daemon startup. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I have the latest rsync running on mandrake official 10. At 2:30 I run rsync --daemon from cron to run an offsite backup for all my customers. At 8:00 I run killall rsync. However my log files show the following: 2004/04/20 02:00:00 [2702] rsyncd version 2.6.0 starting, listening on port 873 2004/04/20 02:00:59 [2707] rsyncd version 2.6.0 starting, listening on port 873 2004/04/20 02:00:59 [2707] rsync: open inbound socket on port 873 failed: Address already in use 2004/04/20 02:00:59 [2707] rsync error: error in socket IO (code 10) at socket.c(395) 2004/04/20 02:02:01 [2719] rsyncd version 2.6.0 starting, listening on port 873 2004/04/20 02:02:01 [2719] rsync: open inbound socket on port 873 failed: Address already in use 2004/04/20 02:02:01 [2719] rsync error: error in socket IO (code 10) at socket.c(395) This same log repeats until the killall is received at 8:00am -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [PATCH] time limit
Ok, now that's just plain ugly.
Here, I just wrote you this...
[EMAIL PROTECTED]
/home/cnwt99>cat runfor
#!/usr/bin/perl
#runfor usage:runfor seconds commandline
#need it to get the correct value for WNOHANG
use POSIX ":sys_wait_h";
#pull the params
($runtime, @commandline) = @ARGV ;
#assemble the commandline
$commandline = join(' ', @commandline) ;
#save the pid to kill
if( $kidpid = fork() ){
#and let it run
sleep($runtime);
#see if it's still there. the "0" looks for children within this
#process group, in case it's completed and the pid's been re-used...
#it happens, though it might be held as a zombie for us... can't
#count on it, though. - if response<0, it's still running
unless(0 < waitpid(0, WNOHANG)){
#kill it
kill(15, $kidpid) ;
}
exit(0);
}else{
#fork and give it our slot
exec( $commandline );
}
[EMAIL PROTECTED]
/home/cnwt99>
Tim Conway
Unix System Administration
Contractor - IBM Global Services
desk:3032734776
[EMAIL PROTECTED]
Jan-Benedict Glaw <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED]
04/20/2004 06:30 AM
To
[EMAIL PROTECTED]
cc
Subject
Re: [PATCH] time limit
On Mon, 2004-04-19 16:28:08 -0400, John Taylor <[EMAIL PROTECTED]>
wrote in message <[EMAIL PROTECTED]>:
> I have written a patch for rsync-2.6.1pre-2 which adds a --time-limit=T
> option. When this option is used rsync will stop after T minutes
> and exit. I think this option is useful when rsyncing a large amount
Okay, nice thing. What about
$ echo "killall rsync" | at midnight
Of couse, you can make that a bit more flexible if you've got several
concurrent rsync running (by using their PID and checking it before
killing it to actually be a rsync instance)...
MfG, JBG
--
Jan-Benedict Glaw [EMAIL PROTECTED]. +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf| Gegen Zensur | Gegen
Krieg
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im
Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
TCPA));
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
signature.asc
Description: Binary data
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Re[2]: temporary file
If you mean with the --whole-file, then in part, it may waste bandwidth, as any change in the files metadata will trigger a whole new send. Like I said, it depends on the nature of your data. For instance, the file you mention as an example - a .tar.gz file, will probably not have any re-useable blocks of data, and will always be resent in toto if sent at all, no no matter the flags. --whole file's main optimization is to avoid reading the whole file repeatedly over the network filesystem. The ratio of file adds to file modifications, and the ratio in speed between your link from source to destination versus the speed of the filesystem, and the processor power available on both ends, determine whether --whole-file is good or bad. Someone smarter than me could probably model it for you, but the best advice for most of us is to take a typical run and do it both ways, and pick the best performer between the two. Also, there will always be some copying if you work from an existing file. Rsync does not, under any circumstance, modify a file in place. It works from the original and the delta to create the new file, and the old file doesn't go away until its replacement is ready to take over. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] MEGA Hospedagem <[EMAIL PROTECTED]> 04/19/2004 09:12 AM To Tim Conway/Denver/Contr/[EMAIL PROTECTED] cc Subject Re[2]: temporary file I guess there's waste of BW because rsync will be used in this case like "cp" It's not just sending differences, and it will send even files that are equal to the ones that already exists. Am I wrong? -- Luis Fernando TC> no. TC> it creates the temporary file, then deletes the old file and renames the TC> temp file to the correct filename. Unless samba won't let you rename a TC> file, (I know windows moves within a filesystem are copy/deletes, at least TC> with cygwin on windows 2000), there's no actual waste there. I'd expect TC> rsync to work pretty well with samba, considering its ancestry. TC> If the samba share is a bottleneck, I'd bet you'd get a big boost by using TC> the --whole-file option... it kind of depends on the nature of your data, TC> the filesystem, and the link between the systems. TC> Tim Conway TC> Unix System Administration TC> Contractor - IBM Global Services TC> desk:3032734776 TC> [EMAIL PROTECTED] TC> I did the following command: TC> rsync -a --delete cpbackup /mnt/backup/ TC> It works well, but the problem is that it seens the "temporary files" TC> (ie: -rwxr-xr-x 1 root root 83623936 Apr 19 2004 TC> .name.tar.gz.yGk7m7* ) are being created on /mnt/backup/ TC> Since it's a SAMBA mounted partition, it waste BW and is slower. TC> Is there a way to specify where I want the temp files to be created? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: temporary file
no. it creates the temporary file, then deletes the old file and renames the temp file to the correct filename. Unless samba won't let you rename a file, (I know windows moves within a filesystem are copy/deletes, at least with cygwin on windows 2000), there's no actual waste there. I'd expect rsync to work pretty well with samba, considering its ancestry. If the samba share is a bottleneck, I'd bet you'd get a big boost by using the --whole-file option... it kind of depends on the nature of your data, the filesystem, and the link between the systems. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I did the following command: rsync -a --delete cpbackup /mnt/backup/ It works well, but the problem is that it seens the "temporary files" (ie: -rwxr-xr-x 1 root root 83623936 Apr 19 2004 .name.tar.gz.yGk7m7* ) are being created on /mnt/backup/ Since it's a SAMBA mounted partition, it waste BW and is slower. Is there a way to specify where I want the temp files to be created? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to:- copy only files less / greater than X size / date?
find paths -options -print |rsync -options -files-from=- source destination The second one could easily be done with a --dry-run, preprocessing the output, and driving a --files-from= rsync with that list. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi there, I think the subject pretty much says it all... Is this possible? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync problems
There are two things that bring that message. One is the inability to chroot. I think non-chrootable operating systems get their rsync configured so it never tries (and therefore, never fails). If you're on a real OS, you might be initiating the daemon as a non-root user (uid in the module doesn't matter). The more common cause, though, is an error in the "path = /whatever" entry for a module. You can't chroot to a place that doesn't exist. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] 1. @ERROR: chroot failed rsync: connection unexpectedly closed (34 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) > ... Any ideas? > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: want to write a patch, need help getting started...
--archive will make it match up all the timestamps. owners, etc, along with the data. If the metadata matches, it won't read the files to verify content, so you might want to add --checksum to force reading, though that will make it take a whole lot longer. If you want it to leave owner alone, do individual options instead of the --archive bundle... --times will do, though you probably want it recursive, too. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Greetings, I have two systems that have identical file hierarchies, except the owners/groups and timestamps differ. All of the files have the same checksums. Is there a way to tell rsync to compare checksums, and if they are the same, simply change the owner/group/timestamp? I really only care about timestamps. This way, once the timestamps are duplicated, rsync can operate more efficiently the next time it is run. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Fwd: EAsy_EA
If you can explain what this has to do with rsync, perhaps we can help you. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] I purschased Medal Of Honor Allied Assault Spearhead, it won't install on -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: source dependend path in rsyncd?
If there were an option to do that, it would be in the man page, don't you think? The solution is to write the modification yourself, or to take the reasonable path and use different module names for each host. If the issue is that you want to use a single script on multiple hosts, make modules in the form [fool-hostname] with paths to /some/path/foo-hostname, and have your script send it to rsync-server::foo-`uname -n`. You may also want to use the "hosts allow" option per module to prevent the wrong server from somehow getting into a module. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi, can I make one rsync url available to several machines, but on the rsync server, direct that url to different directories? for example allow all hosts to access /foo but direct host a to /some/path/foo-a/, host b to /other/path/to/foo-b/ and so on? Thanks. Andreas -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync - copy only new files ( since the last run )
rsync -options `find source -type f -name 'pattern' -newer markerfile -print;touch markerfile` destination Or perhaps, since new files might appear during the process: ++ tosend=`find source -type f -name 'pattern' -newer markerfile -print` rm markerfile ln `ls -tr $tosend |tail -1` markerfile rsync -options $tosend destination ++ That way, you retain your newest file sent as the cutoff. I think there's a touch command to copy timestamps, but it's not handy in my head at the moment, and as you are obviously retaining the old logs anyway, there's no harm in adding a link reference... it doesn't eat much. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] "Roman Kab" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 04/13/2004 11:51 AM To <[EMAIL PROTECTED]> cc Subject rsync - copy only new files ( since the last run ) Hello everyone, Here is what I am trying to accomplish. Summary: Using rsync to push journal transaction files from one server to the hot backup server. Rsync script runs every 10 minutes and pushes the files . On the hot backup server these files are applied by a shell script. A purge job deletes files that have been applied from the hot backup server. Unfortunately rsync pushes missing files back from primary to the hot backup. Question: Is there a way to make rsync pickup where it left of instead of pushing all missing files. I understand that the inteded function of rsync is to sync up the dirs. May be there is a timestamp, flag .. that I can pass. There are times when it is necessary to sync up but that occurs once in a great while. Thanks Roman -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Regarding ownership !!
Holy cow! Now that's a blast from the past. The thread started two years ago... at least, the one he included part of. Since then, I lost that job, spent 14 months unemployed, and got into a new job. Anyway: your set of options is an almost complete specification of the "--archive" or "-a" option. All you lack is "--links" or "-l"... bear that in mind if you'd like to shorten what you have to type. The --owner option is meaningless unless the process at the destination is running with at least an effective UID of root(or 0), as only the root user can "give files away". Looking at your results, I suspect you're sending to an rsync daemon - a very fine option. The default is for an rsync daemon to spawn as UID "nobody". If you want the rsync server to be able to write files to belong to a user besides "nobody", you'll have to specify a UID - either that of the person you want to own the files, or "root" if you're handling multiple uids in a sync, as is undoubtedly the case. If you're opening an rsyncd with root perms, make sure you control it. If you have to give it access to multiple directories in / at the same time, exclude /etc/ /bin, and so on. Better yet, change your process so you can handle different directories under different modules, and put them in chroot jails. A trick around that is to crossmount, if your OS permits it... like say you need to give somebody access to /usr/openv and /oracledata in the same module. make a module with a path "/rsyncout". make the directory /rsyncout and /rsyncout/usr/openv and /rsyncout/oracledata mount /oracledata /rsyncout/oracledata mount /usr/openv /rsyncout/usr/openv Now, those filesystems appear to be right there in their correct relations to /, but hiding in your chroot jail. Like I said, many operating systems don't like that. Most refuse. Some comply and corrupt the filesystem. Some probably explode. :-) You probably just need a destination all by itself, and have a quick, simple, and easy road ahead. I just want to ensure that your'e thinking about security as you open a root access to your system. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Lakshminarayanan Radhakrishnan <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 04/09/2004 09:38 AM To [EMAIL PROTECTED] cc [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED], Lakshminarayanan Radhakrishnan <[EMAIL PROTECTED]> Subject Regarding ownership !! Dear Mr.Tim, Options used in rsync command in our system: rsync --verbose --recursive --update --delete --group --owner --times --perm Eventhough i am using "-owner" option while synching to the mate system. The owner ship from the System A is not restored in the System B. ( System B is destination ). Why the ownership of System A is not restored in System B ? ( eg. ) file in system A :rwxr--r-- lakshmi comp a.c file in system B: becomes, rwxr--r-- nobody nobody a.c Is there any option to restore the permission ? thanks, Lakshmi [EMAIL PROTECTED] wrote: > Sent by: [EMAIL PROTECTED] > 02/05/2002 08:57 AM > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync is slowing down
I wrote a tool to get around a similar problem. In my case, I had to bail on rsync altogether... the hardware was just too unreliable and limited in resources. I've BCCd the current custodian of that tool, who was at least at one time a member of this list. If my old company has no objection to the sharing if my IP and he gets a moment to come up for air amid his drowning in both his and my work, maybe he can post it. If you're going to drive rsync with it, you'll need some mods, as currently, it uses packetized tar heck, you'll need mods anyway, as it's got a lot of bug workarounds to get along with the incompetent Maxtor NAS devices, and the code is so ugly, you may have to rewrite it anyway just to avoid hurting your eyes. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Phil Howard <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 04/07/2004 05:44 PM To [EMAIL PROTECTED] cc Subject Re: rsync is slowing down On Sat, Apr 03, 2004 at 12:23:59PM -0800, Wayne Davison wrote: | You can implement such optimizations on top of rsync using either | excludes or the --files-from option. For instance, if the sending | side maintained an exclude file of old directories that didn't need | to be transferred, you could write a script that would look for | updated items and remove the appropriate exclusion. An exclude list | would have to be grabbed first from the remote side before it could | be used, though. How would the sending side know what directories are "old" for a given receiver? One receiving side may run their update today for an old directory that had one file changed. But another receiving side may not run its update for a few more days or even weeks. This sounds like the sending side needs to keep track of what each different receiver has or doesn't have. That's what I used to do before rsync. -- - | Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ | | (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ | - -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync OS X/Linux
Left to right. The very last path on the commandline is the destination. Don't put any source location there. If you're not certain what will happen with a particular commandline, add "-n" to it, along with at least one "-v", and it'll tell you everything it would do. "man rsync" will yield a wealth of information on the --compare-dest=, --backup-dir=, and other useful options for incremental backups. In the archives, you'll even find a nice application to use rsync to keep up (and restore from) a sort of "snapshot" structure. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Hi, I'm kind of a newbie in development and need to create an incremental backup of my http server on my computer (through ssh). But after reading the manual I had, I afraid of mixing my source and destination. Can somebody give me the right command and help me developing a daily incremental backup script? Thx, Selim. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Communication problem with rsync-2.6.0
The first problem is that you're attempting to use a remote transport on a local transfer. The second problem is that you have put "-e rsh" after filespecs, thus having it interpreted as such. What you have asked rsync to do is to put the contents of "source/", "dest/", and the object "-e", in to the directory "rsh". I'm just guessing that's not what you wanted. A correct cmdline to do what it appears you want would be "rsync -r source/ dest/" . If you have removed a hostname from the source or dest path, you'd want "rsync -r -e rsh source/ desthost:dest/" or "rsync -r -e rsh sourcehost:source/ dest/" , depending on which is the remote. If I read my country codes correctly, you're to be commended for interpreting the documentation as well as you have... I sure couldn't read Polish. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] hello, I have a following problem while using rsync: rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) it was coused by command rsync -r source/ dest/ -e rsh -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to RSYNC from eth1 on PDC-SRV to eth1 on BDC-SRV?
The question is incongruous, in light of the title, but: install the cards. get them recognized and configured. connect them, perhaps with a crossover cable (ethernet nullmodem). configure them with addresses and netmask - since this is a point-to-point, no external routability is needed - I suggest 192.168.0.0/31. Name them, in /etc/hosts, as this isn't going to be dynamic data, nor shared to other systems. Don't name them the same as the primary hostname. Make each the route to the other. Example: pdc has eth1 configured at 192.168.0.0, named pdc-eth1, and bdc has eth1 configured at 192.168.0.1, named bdc-eth1, both with netmask 255.255.255.254. On pdc - "route add host bdc-eth1 eth1" On bdc - "route add host pdc-eth1 eth1" Use the "-eth1" hostname in the source or destination of the rsync commandline. Or, you can leave off the naming and just use raw IP addresses. BTW: You'll find that a crossover cable runs extremely fast, as there is no resource sharing in the transport. Your dedicated 100bT channel may actually be FASTER than the shared gig-e. Another caveat: You mention advanced routing. You may run into an issue wherein your Gig-E interfaces tell each other that they're better routes to the eth1 addresses. THAT's an advanced routing thing, and I'm not certain how to force system's IP stacks to segregate ARPing. Good luck, Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] hours on Google.com searching, and I have not found an answer. I am not highly-skilled at setting up advanced routing, so I am not sure how to accomplish this task and do not know if it is even possible. Any help and advice would be greatly appreciated. Travis L. Bean Systems Administrator Bio-Logic Aqua Technologies Grants Pass, OR - United States -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Timeout question
You're looking at an NFS issue. Rsync doesn't apply timeouts to filesystem access... it treats it like connection establishment. If you were doing it over ssh, it wouldn't count the time for ssh to establish(or return an error) in the timeout time. I can't say for certain on the rsync internal transport - my guess would be that timeout applies even to establishing the TCP session, there. Once the stream is established, timeout counts. I don't think it ever counts in a local->local transfer. By the way, with NFS mounts, add the -W, as it's already going to have to read the whole file over the network, and might as well just read and write it in one step... or does it now reliably notice that a filesystem is NFS and force -W? I hope not, as fast local NFS and slow WAN would probably still be better off using the rsync algorithm. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] "gianluca gattelli" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/31/2004 05:34 AM To [EMAIL PROTECTED] cc Subject Timeout question Hi. I've got 2 Fedora Linux (a Master and a Slave) with RSYNC-2.5.7. I need to keep Master synchronized every 5 minutes (mounting a remote directory with NFS on Slave). It runs correctly. To simulate a problem, I try to disconnect the lan cable. From the shell of Slave: rsync -a -v --timeout=10 --delete /mnt/master/ /local/slave I expect that the process will end after 10 seconds, but it seems frozen. Then, if I reconnect the lan cable, this message is shown: rsync: writefd_unbuffered failed to write 424 bytes: phase "unknown": Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) Why the "--timeout" option doesn't work in these conditions? Thank you in advance. Bye G. ** The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only. If you have received this email in error please notify the system manager or the sender immediately and do not disclose the contents to anyone or make copies. ** eSafe scanned this email for viruses, vandals and malicious content. ** ** -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Failed rsync -- two different files considered up to date
I hadn't thought of that one... I was going to suggest -c (--checksum). Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Greger Cronquist writes: > I've used rsync successfully for several years, syncing between two > Windows 2000 servers using daemon mode, but today I stumbled accross > something peculiar. I'm using cygwin with rsync 2.6.0 at both ends (the > latest available at this date) and I have a file that rsync considers up > to date even though both the md5 and a normal diff show differences. > I've tried calling rsync with several different options, most notably -c > for forcing checksum, but it fails to see a difference between the files. > > Are there any things I should try or information that I can include? All > -vvv gives me is "uptodate". How about -I (--ignore-times)? Craig -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync/cwsync dir spaces on the path
Because spaces are part of the syntax of the rsyncd.conf, you'll need to work around them. Wildcards are outside the concept of modules, as referring to the module is referring to a specific location on the filesystem, not a range of possible choices. You can either use symlinks to give nice clean names, [EMAIL PROTECTED] /home/cnwt99>ls -l mydocs lrwxrwxrwx1 cnwt99 mkgroup- 170 Mar 8 13:49 mydocs -> /c/Documents and Settings/cnwt99/My Documents [EMAIL PROTECTED] /home/cnwt99> or find the short name for the items. C:\cygwin\home\cnwt99>dir /? /X This displays the short names generated for non-8dot3 file names. The format is that of /N with the short name inserted before the long name. If no short name is present, blanks are displayed in its place. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi: First let me say that rsync/cwrsync is amazing ! And sorry ! if I'm bothering you with this question... I need to sync my work files from the WinXP Pro machine to our Solaris one... so far so fine.. My problem is when I specify the path on the rsyncd.conf (cwrsync) I have tried many combinations from: /cygdrive/c/Documents*/rbadilla/My*/clients /cygdrive/c/Documents*/rbadilla/ "/cygdrive/c/Documents and Settings/rbadilla" /cygdrive/c/Documents\ and\ Settings /cygdrive/c/Documents*tings /cygdrive/c/"Documents and Settings/rbadilla" Until now I always get the " chdir /cygdrive/c/XXX failed : No such file or directory " Can you enlight me??? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Root access over ssh?
Please tell me you're just going along with the joke. In case you're not, please immediately remove those lines and rehup your inetd. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] > you might as well also add "opendoorstream tcp nowait root > /bin/sh sh" to your inetd.conf and "opendoor666" to services. Hmmm, thanks for the advice, I tried this, but my rsync still isn't working. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Root access over ssh?
yes You have to have a "uid = 0" in the rsyncd.conf file for that module, since only root can give away files, and also bypass all (except over NFS, etc.) file protections. For (insert diety's name here)'s sake, don't use "/" for a path, and don't leave it un-chrooted, unless you're putting up a honeypot or something. You might want to password-protect that module, too, to inhibit casual unwanted file modification. If you have this [modulename] path = / uid=0 read-only = no In your rsyncd.conf, you might as well also add "opendoorstream tcp nowait root /bin/sh sh" to your inetd.conf and "opendoor666" to services. Maybe you could hide behind excludes, but I wouldn't count on it. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Paul Galbraith <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/25/2004 10:00 AM To [EMAIL PROTECTED] cc Subject Root access over ssh? Is it possible to configure rsync in server mode, to gain access to root protected files, without the user having to log in as root through ssh? I'd prefer to login as a regular user through ssh and access an rysnc server on the host that's running as root. As far as I can tell, however, that's not possible...am I wrong? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Logging from cron
"$rsynccommandline >$logfile 2>&1" is one way - use however many "v"s you need. If you want to accumulate the output of several invocations, "$rsynccommandline >>$logfile 2>&1" If you really want it in syslog "$rsynccommandline 2>&1 |logger $facility.$severity", and make sure you have syslog configured to put that where you want it. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] "T. Coutu" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/23/2004 10:53 AM Please respond to coutu3 To [EMAIL PROTECTED] cc Subject Logging from cron Hello, I've just spent several hours going over several Google searches trying to find a way to configure rsync to log into a file named "/var/log/rsync.log". So far, every instance where I've found someone asking about rsync logging remained unanswered (which is kind of weird in itself). As far as I can tell, the only way to do this is to setup rsync as a daemon process so that it will read an rsync.confg. The problem is that I don't want to run rsync as a daemon process, and am not interested in working from a rsync.confg file if I can avoid it. My rsync command lines are being generated on the fly by a perl program running via cron, and then shelling the command to the OS (linux) and handshaking via SSH. This is working fine, but I need a log to determine what is causing rsync to shutdown before completing the full mirror of the server. Does anyone have any suggestions as to how I can get a log generated from rsync running from the command line? Many Thanks in Advance, Tim -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Long time needed for "Building file list" Any suggestions ?
Good idea find / -ctime -1h |rsync -a --files-from=- / destination No perl needed. You might want mtime instead, though. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] Jim Salter <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/22/2004 04:19 PM To [EMAIL PROTECTED] cc Subject Re: Long time needed for "Building file list" Any suggestions ? > This does bring up one point though. Is there any way to optimize file > list building? It seems like that turns into a huge bottleneck in the > "lots of files" situation. If you already know you're working with a mirror on the other end, and you know when your last sync was, and you're a moderately decent Perl hacker, you can pretty easily hack together a script that will take the output of something like find / -ctime -1h -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: orphan dirs and files with --delete
--force force deletion of directories even if not empty SunOS 5.8 Last change: 26 Jan 20037 User Commandsrsync(1) That should do it. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 [EMAIL PROTECTED] rsync (2.5.[67]) --delete fails on dirs with the w bit cleared. (example below) Rsync will sync a dir with w bit clear, but will not remove it with --delete. This is not a big problem, but it will create situations where there are 'orphaned' files. Has anyone else had this problem? It looks like a change would be needed in robust_unlink (util.c). This function would have to do a chmod on dirs that are locked down before it does the unlink. (syncing as user root doesn't have this problem) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: remote client and server
Since you say "mv doesnt' work like that", I assume you're moving directories from one filesystem to another. rsync acts sort of like cp, or the cp part of a mv across filesystem boundaries, in that it goes item by item. Unlike mv and cp, it doesn't write the incoming file to its eventual name until it's successfully present (or at least until the transfer stops with --partial), but if it gets 2 of 8 items copied over and dies, it doesn't undo the transfer. What you really need is a sort of a custom wrapper for rsync, or cp, or mv, or whatever you choose, that does the copy, shadowing anything overwritten to a temporary backup, then, if the transfer succeeds, remove the source files, else, remove everything sent over, and move back the shadowed items. If you're always sending things that don't already exist on the destination, you can leave out the shadowing... your call. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] I need to move files from one directory to another directory remotely. I can issue an 'mv' command through SSH but I want something that will move all the files that i tell it to move or none, and 'mv' doesnt work like that. Is it possible to use Rsync to do this? I was hoping to invoke rsync from machine A on machine B to machine B. Is this possible ? If you can suggest any other tools or ideas please let me know. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: list of files at the rsync destination
If you must "push" the transfer to the destination, push over the listing
from the run, after you're done.
If you can "pull" the transfer to the destination, you can take the list
directly from the rsync process, and use the output to drive the process
of running the commands in real time, rather than waiting for the rsync
run to finish.
Don't forget your "-v".
If the destination is static, except for the rsync update:
before the run
find destination -type f -print |sort|xargs sum >beforefile
after the run
find destination -type f -print |sort|xargs sum >afterfile
diff beforefile afterfile |awk "/^$destination/{print \$1}" |sort|uniq
|while read file
do
[ -f "$file" ] && yourcommand $file
done
Tim Conway
Unix System Administration
Contractor - IBM Global Services
[EMAIL PROTECTED]
What is the best way to get the list of filenames being transferred to the
destination -- from the destination side? We are not running in daemon
mode. We'd like to run a command on each file on the destination after the
file has completed rsync-ing.
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Backing Up Files I Don't Own
The obvious (and bad) idea is to use a suid rsync on the remote end. Fortunately, rsync notices that, and refuses to act like root unless it was invoked by a root-owned process. Hack it if you want. If you're comfortable with this, write a wrapper on the remote that does a sudo /usr/local/bin/rsync $@ and point to that wrapper with the --rsync-path= option. I think the password prompt will come through stderr so you can respond to it - test it for yourself. I doubt you want to leave a passwordless sudo open, but that may be the only way. The safest (in my opinion) alternative that permits unattended operation is to expose the stuff you want to back up via a rsyncd, read-only, chrooted, password-protected, non-listed root-uided module. If you have confidential information that will be exposed through this module, and your company's policy doesn't permit telnet, (sniffable passwords and uids), you probably don't want to do this. Next is same, but add hosts allow = localhost, and get it through an ssh tunnel. That'll hide the rsync authentication, AND your data. Regardless, don't make the uid:password combo for the module "root:rootspassword". It'll be root access, but highly limited - no point in letting that little hole be a big one. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] CLIFFORD ILKAY <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/17/2004 01:09 AM To [EMAIL PROTECTED] cc Subject Backing Up Files I Don't Own Hi, I need to back up all of /home on a remote server for which I have root access but cannot (and will not) do root logins via ssh. Of course if I attempt to rsync files that I don't own, rsync skips over them. My account is allowed to sudo, if that helps. How can I use rsync to do the following: rsync -av --compress --progress --delete -e [EMAIL PROTECTED]:/home /home/buForSomeRemoteServer -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsyncd without syslog
First, good luck with your dynamic data... it sounds like you might be wanting a distributed filesystem instead, but on to the question: In your rsyncd.conf log file = /dev/null That stops sending to syslog, and throws away the log data (rather than storing it in a file). Oh, and if you hadn't apologized, nobody would have guessed that you weren't a native speaker of English. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] Hi, is it possible to use the rsyncd Daemon without any logging. I would like to make a network synchronization on a specific directory in a small network (10 Hosts) . These synchronisation should happen every 10 secounds. My logfile increases to fast with the logging option therefore it is better to use rsync without any logging. Thanks for answering. Please apologize my bad english. I hope you know what I mean. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: suppressing motd without decreasing verbosity
The simplest solution is to not have the rsyncd demand that clients display the motd. motd is not considered part of verbosity, so the only way to shut it off on the client side is to have the client shut all the way up. +++ [EMAIL PROTECTED] /home/cnwt99>man rsyncd.conf |grep -C 2 "motd file" motd file The "motd file" option allows you to specify a "message of the day" to display to clients on each connect. This usually con- tains site information and any legal notices. The default is no motd file. [EMAIL PROTECTED] /home/cnwt99> +++++++ Short of that, I'm stumped. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] Is there a way to make the rsync client suppress the motd without suppressing other messages when connecting to an rsync server? What I want is to run rsync from cron and have it produce output only when any files have been downloaded or deleted and whenever errors have happened. Otherwise, I want it to be quiet. This doesn't seem to be possible with rsync as of version 2.5.7. When I use the -v option, the motd and file transfers are printed. With either -q or -vq, nothing is printed. When I don't use any of those options, then motd is printed but file transfer is not reported. There doesn't seem to exist an option for reporting file transfers only, or is there something I am missing? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync wont work
There we are. Thanks. OK, you're using ssh, and you can ssh in, so we can assume you're getting in. Now, "ssh [EMAIL PROTECTED]" and then "which rsync" is different from "ssh [EMAIL PROTECTED] which rsync". ssh without a command starts a login shell, and you get your full environment. With a command, you get a very stripped-down environment, which is unlikely to have "/usr/local/bin" in the path. I can state with near unity certainty that rsync -vvvcrlpogtz --rsync-path=/usr/local/bin/rsync /tmp/ [EMAIL PROTECTED]:/tmp will work. Alternately, you could modify your system initialization scripts to get /usr/local/bin in your basic path, or symlink /usr/local/bin/rsync into /bin or /usr/bin. I generally leave the system unmodified, and change my cmdline, but if you're setting up something for newbies to do their own commandlines, you'll probably end up making it idiot-proof. Oh, and you didn't get the email directly because your system wanted me to authenticate myself. I declined to make the effort. I figured you could just read the copy from the list. Good luck, Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: 'Invalid cross-device link' message on sparc
tmp/rsync/unusable_link-dest/dir/foo and dir/foo are on different filesystems. --link-dest= makes hard links - new directory entries pointing at the same inodes. Directory entries don't have any way to specify the device containing the filesystem. It's assumed that it's the same device containing the directory. symlinks can span devices, but they don't maintain a link count on the file, so deleting the original link takes the link count to 0 and frees the data, and also leaves the symlink as a "broken link". If you want to use --link-dest, you will have to point to a place on the same filesystem containing the stuff you're linking. --link-dest=DIR create hardlinks to DIR for unchanged files Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] [EMAIL PROTECTED]:/tmp/rsync% /usr/local/rsync-20040311/bin/rsync -a -v --link-dest=/tmp/rsync/unusable_link-dest source/ dest building file list ... done created directory dest ./ dir/ link /tmp/rsync/unusable_link-dest/dir/foo => dir/foo : Invalid cross-device link -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync wont work
As you may recall from the last time I answered you, please send the same thing, but use 3 "v"s instead of only one. rsync -vvvcrlpogtz . [EMAIL PROTECTED]:/rsync/ Since you don't know which transport you are using, we can read the output from the above command to find out. You are not getting to a remote rsync process, either because the transport (rsh/remsh or ssh) is not getting there (only a failed authentication is possible, from the fact that you're getting a password prompt), or because once it's there, it's not finding the remote rsync binary(or not allowed to execute it). Once we know you're getting a shell open to [EMAIL PROTECTED], we then do (rsh || ssh) (depending on what rsync is using in your case) -l rsync domain.ltd which rsync which will probably give a "not found". find where it is on that system, and try your command again with --rsync-path=/wherever/you/have/rsync/on/domain.ltd/rsync . Repeating the question in exactly the same way, yet again, will not give the information needed to solve your problem. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] "alexus" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/12/2004 12:19 PM To <[EMAIL PROTECTED]> cc Subject rsync wont work can someone tell me what am i doing wrong? d# rsync -vcrlpogtz . [EMAIL PROTECTED]:/rsync/ [EMAIL PROTECTED]'s password: sh: rsync: rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) d# it works fine if i just specify directory instead of remote site, but thats not what i need.. thanks in advance alexus -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Where is my bottleneck?
This seems obvious, but it's easy to forget. Are all the filesystems on both ends locally-attached? If they're NFS or SMBFS, rsync is about a third the speed of SCP, unless you use "-W" to tell it to send the whole file if the timestamp/size don't match... in most cases, anyway. ethernet lan and dialup wan to the remote server, I'd probably go ahead and let it use the rsync algorithm. Try not using "-z"... Actually, maybe you already aren't. Here's a strange one for you to try. I'm sure you're running wide open. Try using --bwlimit= to hold it back a bit. If your network is chattering, backing off the throttle might get you down the track a hair faster. Since everything is inside, try opening up rsh for a while (don't leave it open if you're internet-connected.) on host1 time dd if=/dev/zero bs=1024k count=1024k |rsh host2 dd of=/dev/null That'll give you a good idea of what your network can do. Try tarring up whatever you're sending by rsync, sending it straight to /dev/null. Feed it through dd for a byte count. time tar -cf - whatever |dd of=/dev/null bs=1024k Some controllers don't do very good high-speed/volume I/O. Cache will keep things seeming fast, but when you go past cache, you see the true speed of your disk subsystem. The above test will show you some of that. if read's good, check write time dd if=/dev/zero bs=1024k count=1024k of=/bigfilesystem/fileyouregoingtodelete Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] I'm running rsync via ssh on two machines connected with 100 Mbit/s Ethernet cards (at high speed) via a Linksys switch. It's all right here in a single closet. I'm sending large files (initial transfer, nothing preexists on the destination machine) and seeing transfer rates in the 2 Mbit/s range. The CPUs on both machines are more than 90% idle, and I upped the --block-size to 256 kilobytes. Disk activity is very light. Fedora Core 1, i686. I don't expect 100 Mbit/s by any means, but 2? Is this typical? What am I missing? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE:failed connection(was "rsync")
First - nice subject. On the rsync list, it's good to tell everybody that your question concerns rsync. I mean, we'd never guess that, and just that subject alone gives an indication of the nature of the problem, right? Anyway, the rsync process you're invoking is not making contact with the rsync on domain.ltd. Either you're not successfully establishing a shell connection, or the one you establish isn't finding the rsync binary on the remote. First, "rsync -vvv [EMAIL PROTECTED]:". That'll give you something like this if your transport is rsh. +++ [EMAIL PROTECTED] /home/cnwt99>rsync -vvv [EMAIL PROTECTED]: opening connection using rsh -l ync domain.ltd rsync --server --sender -vvvr . Failed to dup/close : Socket operation on non-socket rsync error: error in IPC code (code 14) at /home/lapo/packaging/tmp/rsync-2.6.0/pipe.c(68) _exit_cleanup(code=14, file=/home/lapo/packaging/tmp/rsync-2.6.0/pipe.c, line=68): about to call exit(14) rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown": Connection reset by peer rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.0/io.c(666) _exit_cleanup(code=12, file=/home/lapo/packaging/tmp/rsync-2.6.0/io.c, line=666): about to call exit(12) [EMAIL PROTECTED] /home/cnwt99> ++ Or like this if your transport is ssh. ++ [EMAIL PROTECTED] /home/cnwt99>rsync -vvv [EMAIL PROTECTED]: opening connection using ssh -l ync domain.ltd rsync --server --sender -vvvr . Failed to dup/close : Socket operation on non-socket rsync error: error in IPC code (code 14) at /home/lapo/packaging/tmp/rsync-2.6.0/pipe.c(68) _exit_cleanup(code=14, file=/home/lapo/packaging/tmp/rsync-2.6.0/pipe.c, line=68): about to call exit(14) rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown": Connection reset by peer rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.0/io.c(666) _exit_cleanup(code=12, file=/home/lapo/packaging/tmp/rsync-2.6.0/io.c, line=666): about to call exit(12) [EMAIL PROTECTED] /home/cnwt99> ++ That "opening connection" line is the key. It'll be like the above, or Try a connection using the shell command up to the "rsync". If that works, try that same command, ending in "which rsync". If it's not finding the remote rsync, find it, and add it with "--rsync-path=" to your rsync commandline, or symlink it in somewhere on the stripped-down remote shell $PATH. Good luck. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] alexus <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/09/2004 09:54 PM To [EMAIL PROTECTED] cc Subject rsync d# rsync -vcrlpogtz . [EMAIL PROTECTED]:/export/home/rsync/ [EMAIL PROTECTED]'s password: sh: rsync: rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) d# Can someone tell me what am I doing wrong? Thanks - -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Feature Request: Multiple Streams
Oh, that. There was a lot of talk about it, but it hasn't happened within rsync. I ended up writing my own client-server model utility in perl. We had a master copy of a distribution of EDI tools and views - 170GB or so in a couple million files, as I recall, and we had to keep up around 20 identical copies of it all around the world. I didn't dream of trying to implement the rsync algorithm. It worked strictly on timestamps, sizes, and filetypes, comparing the list from the master with the list from the replica. It made a very lightweight process for each replica, and the generation of the list for the master was done only once per sync. If your changes are mostly new files instead of small changes in large files, it might be what you need. If Tim Renwick is still monitoring this list, maybe he could tar it up and pass you a copy. It'd definitely require some porting for a new environment, unless you're replicating a Maxtor MaxAttach 4000 to others like it, using a Solaris box to handle the master replication tasks. Fortunately, it's commented out the wazoo, so to speak(which made it relatively painless for Philips to lay me off). > It would be nice to have it read the data once, and then sync it to all > of the destinations once. IIRC, there was a move to do this at some > point. Am I right? Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Feature Request: Multiple Streams
for source in source1 source2 source3 do rsync -options $source destination:$source & done wait adapt as needed. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] "Robertson, Jason V" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/09/2004 02:09 PM To <[EMAIL PROTECTED]> cc Subject Feature Request: Multiple Streams Are there any plans to add support for multiple stream copying (BBFTP and GridFTP do this, though only for a single file) to rsync? Thanks, Jason -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync without a "data pool"
You can rsync back over the same data, and only the changes will be brought over. If you're looking for an incremental backup sort of a deal, use the --backup-dir option, which will save changed files over to a new directory, so you can reach back in history. The ability to roll the filesystem back to a specific data, including not having things there that were added later, is a bit stickier, but there are professional packages to handle that. ADSM(tradename has been changed to "TSM" now) is particularly sweet in that regard, and I hear the Veritas NetBackup is nice, too. Tim Conway Unix System Administration Contractor - IBM Global Services [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to rsync only directories, no files inside ?
$)CRsync doesn't have a "-type" qualifier, but as I understand it, it now
includes a "--files-from=" option.
find {path[s]} -type d -print |rsync --perms --owner --group --times
--links --files-from=- {path[s]} {destinationhost}:
Actually, I've just learned that --files-from turns off --recursive in
--archive, so
find {path[s]} -type d -print |rsync -a --files-from=- {path[s]}
destinationhost:
Or, if you're putting them in a different place
(cd {rootpath}; find {path[s]} -type d -print) |rsync -a --files-from=-
{rootpath} {destinationhost}:{destinationpath}
Incidentally, kudos to whoever's doing the man pages now. This feature
was suggested but unimplemented last time I used rsync, and now it's
clearly explained, in detail, with useful examples.
Tim Conway
AIX System Administration
Contractor - IBM Global Services
[EMAIL PROTECTED]
Steven Shiau <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED]
02/29/2004 11:11 AM
Please respond to
steven
To
[EMAIL PROTECTED]
cc
Subject
How to rsync only directories, no files inside ?
Hi.
I need to create a lot of directories, just the same with remote's /var,
but I only need the directories, the files inside those sub-directories in
/var/ are not necessary for me.
I know "find+cp" can do, but I would like to use the archive mode, i.e. I
need the original permision, owner, group...
Can I make it using rsync ? If so, please tell me how.
Thank you in advance.
--
Please AVOID distributing documents in WORD, EXCEL or POWERPOINT format.
[Chinese Big5] tky-Xs_$xOWORD, EXCEL [EMAIL PROTECTED] POWERPOINTL+cRn\?dP.
See http://www.fsf.org/philosophy/no-word-attachments.html
and http://www.cyut.edu.tw/~ckhung/published/997nodoc.shtml
--
Steven Shiau
E-mail: steven _at_ nchc.org.tw; steven _at_ stick.idv.tw
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: exclude everything and include directories
cat listofwhatyouwant |while read item do rsync -options "$item" "destination:`dirname $item" done if you want each directory and below, a recursive goes into the options (explicitely with "r" or implicitely with "a"). I used to have to do something similar to that all the time, as we were using sub-functional NAS devices (anybody ever hear of Maxtor?) and couldn't complete a sync in a single run - makes --delete nearly useless. BTW: Martin, Tridge, Wayne, et. al. - I'm back! That was way too long of a "vacation". Tim Conway AIX System Administration Contractor - IBM Global Services [EMAIL PROTECTED] Don Shesnicky <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 02/26/2004 03:20 PM To [EMAIL PROTECTED] cc Subject exclude everything and include directories Anyone? Even a yea or nea at this point would be appreciated. Don - I'm trying to update some laptops from a large server directory for an EDA app. Most of the time I just rsync the entire directory and exclude the odd item. In this case it's the reverse where I want to exclude everything but include only the odd directory. I thought it'd be easy but can't seem to get the exclude then include options correct. It seems that when you exclude everything you then need to include file by file. Can anyone point me in the right direction? Don -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
goodbye
Since I get so many direct inquiries, I feel obligated to mention that I just got laid off, so am losing this address. Bye. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
RE: rsync problem behind a router
I was afraid of that. I'd hoped that it would show more. Next thing i'd try would be to, on the remote end, write a rsync wrapper script, that starts rsync inside a strace (or truss, or tusc, or whatever your OS uses), directing it to follow forks, redirecting STDERR to a file. Do another super-short run, using --rsync-path= to point at the wrapper, then scan down that trace output. If you don't see a solution right away, post that output here. If we find that the rsync process suspends itself waiting for io, it's probably a transport thing.Rsync is known for revealing weaknesses in TCP/IP implementations, though I wouldn't expect to see it in small jobs. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" "Chad Moss" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 12/18/02 03:48 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: <[EMAIL PROTECTED]> Subject:RE: rsync problem behind a router Classification: OK...I tried with the -vvv on one file (an index.php file). I got: stdin: is not a tty server_sender starting pid=14373 make_file(1,index.php) expand file_list to 4000 bytes, did move recv_file_name(index.php) received 1 names recv_file_list done get_local_name count=1 /home/backup/mercury/ recv_files(1) starting send_file_list done send_files starting generator starting pid=1494 count=1 recv_generator(index.php,0) generate_files phase=1 A couple of times, it did not even get that far...I got: stdin: is not a tty server_sender starting pid=14432 make_file(1,index.php) expand file_list to 4000 bytes, did move recv_file_name(index.php) received 1 names recv_file_list done get_local_name count=1 /home/backup/mercury/ generator starting pid=1497 count=1 recv_generator(index.php,0) generate_files phase=1 recv_files(1) starting send_file_list done send_files starting -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 18, 2002 12:08 PM To: Chad Moss Cc: [EMAIL PROTECTED] Subject: Re: rsync problem behind a router Chad: From the output you have so far, there's not much to go on. Can you try a run with "-vvv"? That's more likely to reveal exactly what it's failing on, though (AFAIK(IIRC)) it won't tell you about stuff on the remote side. I had to write a wrapper to start rsync in a truss -f for part of my debugging. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" "Chad Moss" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 12/17/02 04:57 PM To: <[EMAIL PROTECTED]> cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync problem behind a router Classification: I have used rsync on many systems, and never had a problem. I am stumped on what to do with this? I have a box behind a LinkSys router and I can not "push" or "pull" data to or from it from anywhere? When I try, it logs into the remote server, gets the file list and just stops. The remote server shows a process running until I control-c out of it?when I do that, I get: rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) rsync error: received SIGUSR1 or SIGINT (code 20) at main.c(785) The command I am using is: rsync -avz -e ssh [EMAIL PROTECTED]:/home /home/backup/servername/ I have forwarded ports 22 and 873 to that machine, and I have tried putting it in the DMZ. The results do not seem to change. Thanks. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync problem behind a router
Chad: From the output you have so far, there's not much to go on. Can
you try a run with "-vvv"? That's more likely to reveal exactly what it's
failing on, though (AFAIK(IIRC)) it won't tell you about stuff on the
remote side. I had to write a wrapper to start rsync in a truss -f for
part of my debugging.
Tim Conway
[EMAIL PROTECTED] reorder name and reverse domain
303.682.4917 office, 303.921.0301 cell
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, caesupport2 on AIM
"There are some who call me Tim?"
"Chad Moss" <[EMAIL PROTECTED]>
Sent by:
[EMAIL PROTECTED]
12/17/02 04:57 PM
To: <[EMAIL PROTECTED]>
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject:rsync problem behind a router
Classification:
I have used rsync on many systems, and never had a problem. I am stumped
on what to do with this?
I have a box behind a LinkSys router and I can not "push" or "pull" data
to or from it from anywhere?
When I try, it logs into the remote server, gets the file list and just
stops.
The remote server shows a process running until I control-c out of it?when
I do that, I get:
rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229)
rsync error: received SIGUSR1 or SIGINT (code 20) at main.c(785)
The command I am using is:
rsync -avz -e ssh [EMAIL PROTECTED]:/home /home/backup/servername/
I have forwarded ports 22 and 873 to that machine, and I have tried
putting it in the DMZ.
The results do not seem to change.
Thanks.
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Problem with absolute symbolic links
I haven't seen an answer to this, so I'll get the part I can. You haven't overridden the default for "use chroot", which is "yes". Thus, you can't get the things from outside your module. In fact, rsync explicitely ignores symlinks pointing out of the module unless use chroot = no. ++ use chroot If "use chroot" is true, the rsync server will chroot to the "path" before starting the file transfer with SunOS 5.8 Last change: 12 Feb 19993 Standards, Environments, and Macrosrsyncd.conf(5) the client. This has the advantage of extra protection against possible implementation security holes, but it has the disadvantages of requiring super-user privileges and of not being able to follow symbolic links outside of the new root path when reading. When "use chroot" is false, for security reasons symlinks may only be relative paths pointing to other files within the root path, and leading slashes are removed from absolute paths. The default for "use chroot" is true. ++++++++++ I can't address your first concern. but this handles the second. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" "Kevin Minder" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 12/13/02 11:44 AM To: <[EMAIL PROTECTED]> cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Problem with absolute symbolic links Classification: I'm trying to synchronize a particularly troublesome directory structure on a SunOS box to a PC. I really want all symbolic links resolved to real files on the PC when everything is done. I can't seem to find any combination of parameters that will accomplish this. It boils down to two problems. 1. It seems like that even if I exclude a symbolic link that is a directory it is not excluded. 2. It seems like copy-links and copy-unsafe-links don't work for symbolic links to directories outside the source tree. Is this the correct behavior or am I doing something wrong. Any input is greatly appreciated. Here is my rsyncd.conf file... --- motd file = /etc/rsyncd.motd log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid uid = kminder gid = svrtech [test] path = /home/kminder/rsync/test comment = test read only = yes list = yes --- Below is a simplified source structure on the unix side (source). Note the .rel symlink and the .abs symlink. --- /home/kminder/rsync> ls -al outside test outside: total 16 drwxr-xr-x 2 kminder g9044096 Dec 13 11:37 . drwxr-xr-x 5 kminder g9044096 Dec 13 11:29 .. -rw-r--r-- 1 kminder g904 15 Dec 13 11:37 file test: total 24 drwxr-xr-x 3 kminder g9044096 Dec 13 11:36 . drwxr-xr-x 5 kminder g9044096 Dec 13 11:29 .. lrwxrwxrwx 1 kminder g904 27 Dec 13 11:36 .abs -> /home/kminder/rsync/outside lrwxrwxrwx 1 kminder g904 3 Dec 13 11:34 .rel -> dir lrwxrwxrwx 1 kminder g904 9 Dec 13 11:36 absfile -> .abs/file drwxr-xr-x 2 kminder g9044096 Dec 13 11:34 dir -rw-r--r-- 1 kminder g904 10 Dec 13 11:32 file lrwxrwxrwx 1 kminder g904 4 Dec 13 11:33 link -> file lrwxrwxrwx 1 kminder g904 4 Dec 13 11:33 linklink -> link lrwxrwxrwx 1 kminder g904 9 Dec 13 11:35 relfile -> .rel/file --- I'm trying for the following structure on the pc (cygwin ls -al). Note that all links should be resolved. I want the duplicate files. --- drwxr-xr-x+ 4 kminder None 4096 Dec 13 11:36 . drwxrwxrwx+ 14 Administ Administ 8192 Dec 13 11:20 .. drwxr-xr-x+ 2 kminder None0 Dec 13 11:34 dir -rw-r--r--1 kminder None 10 Dec 13 11:32 file -rw-r--r--1 kminder None 10 Dec 13 11:32 link -rw-r--r--1 kminder None 10 Dec 13 11:32 linklink -rw-r--r--1 kminder None 14 Dec 13 11:34 relfile -rw-r--r--1 kminder None 14 Dec 13 11:34 absfile --- When I execute the command that I think should accomplish this... D:\sync>rsync -rptgDvP --port=34343 --copy-links --copy-unsafe-links --exclude .rel --ex
Re: SPAM on List...
One word of warning on assuming all html mail is bad. Some of us are on corporate email systems, subject to pointless arbitrary changes to our settings. About a year ago, my preferences in Lotus Notes (a bad database program masquerading as a worse mail program) were modified to have me send mail to the internet in html format. I didn't even know it was happening, until i started getting nasty messages from people's mail systems saying my mail was being rejected because it contained html formatting, on which point I threw myself a message to my home address to verify it, and searched out the preferences setting to turn it back to text ...Point being, if we do decide to reject all html-formatted email, let's have it bounce with explanation, not just fall into a black hole. It saddens me to know that blocking all of Korea is the right thing to do. Maybe if more things like this http://slashdot.org/article.pl?sid=02/12/06/1554227&mode=thread&tid=133 happen, things will change. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Christopher Sawtell <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 12/10/02 03:08 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: SPAM on List... Classification: Greetings list admins I'd just like to add that as almost all the SPAM I receive is HTML, a very effective way to get rid of a very large fraction of spam would be to refuse all HTML posts. I've not seen any genuine posts to this list which have been web pages. -- Sincerely etc., Christopher Sawtell -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: cp(1) -n option for rsync?
You could find on the source (exclude directories), and use that output for --exclude-from=. That would be the one case where include/exclude patterns would be easy - no unexpected matches or misses. Yeah, I like that one best. to sync remotehost:/path/to/dir to /dir +++ cd /dir find . \! -type d -print >/tmp/excludefordirfile rsync -a --exclude-from=/tmp/excludefordirfile remotehost:/path/to/dir/. . rm /tmp/excludefordirfile +++ If the dir is small, you could probably even just +++ cd /dir rsync -a --exclude="`find . \! -type d -print`" remotehost:/path/to/dir/. . +++ , but i'm never quite certain what a shell is going to do with my backslashes and quotes, without testing it on the system running the code, and the proc using the interim file is less likely to choke on commandline length. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Sander van Zoest <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 12/06/02 12:53 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:cp(1) -n option for rsync? Classification: Hi, I looked in the archives and documentation and didn't notice anything of this. The closest thing I found was -update, but that doesn't do exactly to what I would like. I would like to be able to use rsync to mirror some directories, but to explicitly *not* override any files that already exist on the other side. Sort of like a file system "append", if you will. The cp(1) command on FreeBSD has the option -n: -nDo not overwrite an existing file. (The -n option overrides any previous -f or -i options.) I was wondering if I could somehow accomplish this doing an rsync between two machines? I'd like to avoid to have to do an test -e over ssh to see the file exists before copying the file accross. Any info would be appreciated. Thanks, -- Sander van Zoest +1 (619) 881-3000 Yahoo!, Inc. [EMAIL PROTECTED] <http://www.yahoo.com/> <http://sander.vanzoest.com/> -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
RE: RFE: using rsync as a backup tool (preserve access time & com pressdestination files) ?
The way gnutar "preserves" atime is by noting it before the read, and setting it back after the read, thus wiping out a legitimate setting of atime occuring during that interval. Yeah, the netapps mess with unix times. Did you notice that mtime and ctime always match? Now that I know you're on a netapp, though, your problems are solved. Snapshot and sync from the snapshot, then expire the snapshot if your data is rapidly-changing, so it doesn't hold a bunch of old space. Also, I just mounted up a netapp readonly (on a system also mounting same directory readwrite elsewhere). even though it's readonly, the netapp sees the read and updates atime... however, I still like the snapshot idea. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Gilles-Eric Descamps <[EMAIL PROTECTED]> 11/15/02 03:50 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:RE: RFE: using rsync as a backup tool (preserve access time & com press destination files) ? Classification: > From: [EMAIL PROTECTED] [mailto:tim.conway@;philips.com] > Sent: Wednesday, October 30, 2002 10:53 AM > > It's not up to the application whether atime gets updated. > That's like > complaining about find making your hard drive light flash. > The only thing rsync could do would be to note the atime > before reading > the file, then falsely set it back to what it was, after > reading the file, > and hope that it wasn't set to something else in the > interim... a kludge > at best. Well, gnu tar provides a "--atime-preserve". All applications which backup filesystems (legato, quickrestore, budtools) preserve times on an application level. A backup tool is supposed to bypass normal filesystem access as it's supposed to be transparent. > Why not mount the filesystem on an alternate mountpoint, noatime or > readonly? On AIX, you can just mount the dir wherever. In sun, and > apparently Linux, nfs export it, only to localhost, if you > like, and mount > it readonly. As Chef Tell used to say, "very simple, very easy", and > legitimate setting of atime can continue unhindered. Because that does not work. I just tried it. The file server is a NetApp box running a proprietary OS (not unix, nor windows). mounted the NFS filesystem with noatime & readonly, but guess what, when you read a file, the box updates the read time -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Speed problem
Here's one of my setups. It's invoked from inetd.
+
Tools@timsync
/home/Tools/newsync/clients/sparetool>grep rsync /etc/inetd.conf
/etc/services ;cat /etc/rsyncd.conf
/etc/inetd.conf:rsync stream tcp nowait root/usr/bin/rsync
rsyncd --daemon
/etc/services:rsync 873/tcp rsyncd # rsync
daemon
log file = /var/tmp/rsyncd.log
pid file = /var/run/rsyncd.pid
[master1]
path = /mastertoolservers/master1
refuse options = checksum
read only = yes
use chroot = no
uid = Tools
gid = Tools
ignore nonreadable = yes
[master2]
path = /mastertoolservers/master2
refuse options = checksum
read only = yes
use chroot = no
uid = Tools
gid = Tools
ignore nonreadable = yes
[admin]
path = /mastertoolservers/master2/admin
refuse options = checksum
read only = yes
use chroot = no
uid = Tools
gid = Tools
ignore nonreadable = yes
[incoming]
path = /users/Tools/incoming
read only = no
use chroot = no
uid = Tools
gid = Tools
list = no
Tools@timsync
/home/Tools/newsync/clients/sparetool>
+
Here's a little script I crapped together to fire one up in any arbitrary
site where I don't have root. An idling rsyncd doesn't eat much cpu or
ram. I just reference it in the crontab for my user, and there's always
one waiting for me.
+
#!/bin/sh
PATH=/bin:/usr/bin:/usr/sbin:/sbin:/etc:/cadappl/encap/bin
export PATH
pidfile=$HOME/.rsyncd.pid
logfile=$HOME/.rsyncd.log
configfile=$HOME/.rsyncd.conf
[ -f "$pidfile" -a -s "$pidfile" ] && ps -p `cat "$pidfile"` |grep rsync
>/dev/null && exit 0
{
echo "log file = $logfile
pid file = $pidfile
[cadappldist]
lockfile = /var/tmp/rsyncd.cadappldist.lock
max connections = 2
path = /cadappldist
use chroot = no
read only = yes
uid = Tools
gid = Tools
list = yes
[cadappldistrw]
lockfile = /var/tmp/rsyncd.cadappldistrw.lock
max connections = 1
path = /cadappldist
use chroot = no
read only = no
uid = Tools
gid = Tools
list = no" >$configfile
rsync --daemon --port=4024 --config=$configfile
}&0 2>&1 &
+
There'a a lot more useful info in the man pages. examine "--port=" and
"--daemon", and maybe "--no-detach" in rsync(1), and read rsyncd.conf(5)
all the way through. You can have password authentication, exclusions,
parameter control... lots of stuff.
Good luck.
Tim Conway
[EMAIL PROTECTED] reorder name and reverse domain
303.682.4917 office, 303.921.0301 cell
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, caesupport2 on AIM
"There are some who call me Tim?"
[EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
11/13/02 12:45 PM
Please respond to uwp
To: Tim Conway/LMT/SC/PHILIPS@AMEC
cc: [EMAIL PROTECTED]
<[EMAIL PROTECTED]>
Subject:Re: Speed problem
Classification:
On Wed, 13 Nov 2002 [EMAIL PROTECTED] wrote:
> I agree, rsh as root is bad. I wouldn't suggest that. I'm talking
about
> running "rsync --daemon", using /etc/rsyncd.conf to control the form of
> the access. It's pretty good for reading, and mostly works for writing.
Do I get you right ? You don't need any transport mechanism, rsync can
to everything by itself ? I thought rsh or ssh is a must. Can you give an
example how to do it ?
Thank you !
Mermgfurt,
Udo
--
Udo Wolter | /"\
email:[EMAIL PROTECTED] | \ / ASCII RIBBON CAMPAIGN
www: www.dicke-aersche.de | XAGAINST HTML MAIL
dark:[EMAIL PROTECTED] | / \
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: getaddrinfo: Host not found problem
I'm sorry. I guess i didn't read your question very well. I think chroot when i hear resolution problems. I think your problem is IPv6-related. As I understand it, "::" is used like "." is in IPv4. Any v6-ers out there who can help out? Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Randy Kasha <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 11/13/02 01:06 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:getaddrinfo: Host not found problem Classification: Tim: The rsync client is on an AIX 4.3.3 pwr3 platform (this is where I am having the problem with the "Host not found"). Your recommendations would only apply to a Linux platform (I think). Although the rsync server is running on a Linux platform, the rsync client never actually contacts it (I get the same error message whether the server is running or not). Any other ideas? Thanks for replying. Randy Randy: I suspect it's running chrooted, and can't see the necessary items to resolve names. Fastest is to turn off chroot. You may be able to get by with just turning off "hosts allow" and/or "hosts deny". It seems that maybe an /etc in the root of the module will take care of it, or so i think i remember reading. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Randy Kasha <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 11/13/02 09:20 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:getaddrinfo: Host not found problem Classification: Hello, I am trying to use rsync (version 2.5.5) in a server client model to distribute software files. When I kick off the rsync client on an AIX 4.3.3 pwr3 platform (machine grp4c), I get the following error message. We are in a real bind to get this protocol going; any help/insight/suggestions would be "greatly" appreciated. rsync: getaddrinfo: grp4c 873: Host not found rsync error: error in socket IO (code 10) at ../clientserver.c(97) _exit_cleanup(code=10, file=../clientserver.c, line=97): about to call exit(10) I can rsh/ping/telnet to grp4c (the server machine - linux) from the AIX client machine with no problems. The rsync client never actually contacts the server at all (when I stop the server on grp4c, I get exactly the same message on the client). Thanks. Randy -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- Randy Kasha WesternGeco DP Global Support Phone: 713 689 2327 -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: getaddrinfo: Host not found problem
Randy: I suspect it's running chrooted, and can't see the necessary items to resolve names. Fastest is to turn off chroot. You may be able to get by with just turning off "hosts allow" and/or "hosts deny". It seems that maybe an /etc in the root of the module will take care of it, or so i think i remember reading. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" Randy Kasha <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 11/13/02 09:20 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:getaddrinfo: Host not found problem Classification: Hello, I am trying to use rsync (version 2.5.5) in a server client model to distribute software files. When I kick off the rsync client on an AIX 4.3.3 pwr3 platform (machine grp4c), I get the following error message. We are in a real bind to get this protocol going; any help/insight/suggestions would be "greatly" appreciated. rsync: getaddrinfo: grp4c 873: Host not found rsync error: error in socket IO (code 10) at ../clientserver.c(97) _exit_cleanup(code=10, file=../clientserver.c, line=97): about to call exit(10) I can rsh/ping/telnet to grp4c (the server machine - linux) from the AIX client machine with no problems. The rsync client never actually contacts the server at all (when I stop the server on grp4c, I get exactly the same message on the client). Thanks. Randy -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Speed problem
I agree, rsh as root is bad. I wouldn't suggest that. I'm talking about running "rsync --daemon", using /etc/rsyncd.conf to control the form of the access. It's pretty good for reading, and mostly works for writing. Oh, on our security - no ssh, but rsh is ok for root. I'm also exaggerating a bit on the reason we don't have ssh. I just can't convince the decision makers that it is important to have it and that rsh is leaving them buck naked to being 0wnz0r3d. Oh, well, it makes it easy for me to get root in an emergency, and so far, we can trust everybody on our network. Tim Conway [EMAIL PROTECTED] reorder name and reverse domain 303.682.4917 office, 303.921.0301 cell Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, caesupport2 on AIM "There are some who call me Tim?" [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/11/02 01:45 PM Please respond to uwp To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:Re: Speed problem Classification: On Mon, 11 Nov 2002 [EMAIL PROTECTED] wrote: This would be an option, but doing rsh with a root account is not possible (couldn't get it to work, I haven't found any way to do it for root and I really want to do it with the root account because I wanna have the same file, directory and user permissions on the files) and also not quite recommended... (.rhosts ??? Sheer horror ! ;-)) Maybe it is the encryption but I use ssh otherwise too and can get on the same line results upto 12 MB/s (blowfish) and even 20 MB/s (arcfour) without any loss of speed. The funny thing is, it seems to happen only after a short while. The first 5 minutes seem to be going good, almost 18 MB/s (also arcfour which means, this is very similar) and then it goes down. It never goes up again, even when a new file get transferred, but it starts at 18 MB/s when I start the complete rsync again. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
