Re: rsync doesn't exit (is hanging) in script, but not on command line!
Hi, I don't know much about perl, but I can suggest doing a little upgrading. Perl is up to v5.8.8 Rsync is up to v2.6.7pre3 Your version of rsync is years old and is vulnerable to multiple security advisories. Thanks, Gian I have two identical (HW&SW) linux servers in my network. On both there is a user called qipadmin, which can access over SSH to each other. I wrote a perl script, which should copy files with rsync from one server to the other. Below is the rsync command in the script. The script is invoked by the root user. system "su - qipadmin -c 'rsync -e ssh -p $otherip:/opt/named/current/conf/db.* /opt/named/current/conf --exclude *.jnl*'" rsync copies the files but the problem is, that the rsync command never exits (is hanging) and so the script cannot continue (unless I do manual kill of the rsync process): ps -ef | grep rsync root 21290 21259 0 10:54 ?00:00:00 su - qipadmin -c rsync -e ssh -p 10.10.1.242:/opt/named/current/conf/db.* /opt/named/current/conf --exclude *.jnl* qipadmin 21291 21290 0 10:54 ?00:00:00 rsync -e ssh -p 10.10.1.242:/opt/named/current/conf/db.* /opt/named/current/conf --exclude *.jnl* qipadmin 21384 21291 0 10:54 ?00:00:00 rsync -e ssh -p 10.10.1.242:/opt/named/current/conf/db.* /opt/named/current/conf --exclude *.jnl* On the command line I can do the following without any problems (rsync exits successfully): su - qipadmin -c 'rsync -e ssh -p 10.10.1.242:/opt/named/current/conf/db.* /opt/named/current/conf --exclude *.jnl*' What is the difference between system "" (perl) and (shell command line)? Or is there a bug in rsync? additional infos: - rsync version: 2.5.7 - kernel: 2.4.32 - perl: 5.8.0 I hope there is somebody who can help me. I'm very frustrated. ___ Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html rsync mailing list rsync@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync help needed...
Hi Isk, Please see embedded. Gian lsk wrote: Hello, I was reading your posts about RSYNC. We have a massive Oracle schema lots of datafiles about 750 GB size. We do rsync datafiles from source to target server but everytime we cleanup the datafiles on the target server and do rsync every 2 weeks. What do you mean by "cleanup the datafiles on the target server"? Are you editing files on the target server? On the target side mostly the datafiles will be same but on source we might have added few datafiles or made some changes in data and as such the size of schema will increase every time since we add new datafiles. If we leave the old datafiles at the target end and rsync will it be faster ? If the target is not 100% different from the source, yes, keep it! Rsync will upload the differences of the two. If 99% of the file is the same, you just saved 99% of your transfer time. :) See timestamps and checksum in the manual to see what option you need to use in your schema. Checksum may take some time with 3/4 TB of data :-/ What rysnc command we need to use without cleaning the old files so in the old files only the changes will be copied and new files also needs to be rsynced. So you want to update files and transfer new files? How about: rsync -e ssh -avz /source/data/files [EMAIL PROTECTED]:/data/files This command works over ssh (for encryption), uses an archive mode (see manual for details), spits out information verbosely, and compresses files during transfer to speed it up. If you would like to watch it happen, add --progress for cool status view. What is the rsync command to be used ? Thanks, lsk. -- View this message in context: http://www.nabble.com/Rsync-help-needed...-t1170765.html#a3075671 Sent from the Samba - rsync forum at Nabble.com. Is this what you were looking for? Gian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync 2.6.7pre1 is now available
How does this function work? I have not seen it discussed on the mailling list. Thanks, Gian On Wed, Feb 08, 2006 at 09:07:13PM -0800, Wayne Davison wrote: Rsync version 2.6.7pre1 is now available for release testing. Please give it a try and send email to the regular mailing list with any questions, comments, bug reports, etc. Hi, The NEWS didn't mention the new functionality allowing to take into account renamed files and not retransfer them. Will it be included in 2.6.7? Thanks, ___ Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html rsync mailing list rsync@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: sync 54 million files, tuning rsync?
How about a shell script? Scan the directories and start "x" instances of rsync at a time, when one finished, have the script fire up another, until the list of directories is complete..? Gian The design is outside of my control. It's not a regular replication it's a migration from one netapp nfs appliance to another. (And no, I can't use the wonderful block-level replication they provide due to some other complex reasons). I'm wondering if there is a way to tell rsync, go ahead, take each dir off the top level and fire up an independent rsync to do this. It seems it's not using all my cpu, memory, or bandwidth (on either side) to accomplish the task. I'm not sure where I'm being throttled, except maybe the NFS client on the Solaris box not being very efficient (Solaris 10, so it *should* be). --- Frank Hamersley <[EMAIL PROTECTED]> wrote: -Original Message- From: [EMAIL PROTECTED] Behalf Of Dirk Schenkewitz Sent: Tuesday, 14 February 2006 10:32 PM To: rsync@lists.samba.org Subject: Re: sync 54 million files, tuning rsync? (offlist) Hi Jerry, I reply offlist because I'm everything but an expert, so you might get a better answer from someone else. Anyway, perhaps it would be good if you post to the list what you're already doing. On Tuesday, 14. February 2006 03:31, Jerry wrote: I'm trying to sync up 54 million files. I can break it down into different applications, but I still have to accomplish 17 million files in one "chunk" if possible. [snip] Jerry, Given you have 54 million entries have you ever considered using a genuine DBMS (with replication) rather than pounding the filesystem to death? Cheers, Frank. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html rsync mailing list rsync@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Info about "File list size" in --stat
My guess is this is the size (in bytes) of the file sent first used to check ownership, mode, permissions, *size* and modtime for syncronization. Rsync uses this information to decide whether or not to update/upload a file or directory. Just a guess though! I couldn't find any documentation on this myself. --Gian Hi! I have a very short and simple question: When using the --stats option in rsync, see result below, what does the term: "File list size: 46" mean? As "Number of files" is 1, I am sure it does not reflect number of files copied. But what is its significance? Number of files: 1 Number of files transferred: 1 Total file size: 51380224 bytes Total transferred file size: 51380224 bytes Literal data: 3556096 bytes Matched data: 47824128 bytes File list size: 46 I am sorry if this has been asked before. I have been trying to search samba.org for the answer, but not found anything. Thanks for your feedback! Best regards, Richard Taubo -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
How many rsync over ssh simultaneous backups?
Hello, I am trying to estimate the number of users I can hold on my rsync backup server if all users are connecting at about the same time. I am using rsync over ssh with 256bit-aes encryption. Here are my average server specs (from least powerful to most powerful): 2.8GHz P4 1GB Ram 1TB SATA w/ RAID 10 for 500GB usable space Kernel 2.4.32 with PaX-Enabled (causes moderate preformance decrease). hdparm shows: cache reads at 932 MB/sec buffered reads at 50 MB/sec - 3GHz P4 w/ 1MB cache 2GB Ram 1.5TB SATA w/ RAID 10 for 750GB usable space Kernel 2.4.32 with PaX-Enabled (causes moderate preformance decrease). hdparm shows: cache reads at 2,080 MB/sec buffered reads at 107 MB/sec All other servers specs are between the two above. Most clients are uploading <100MB of data @ 320kbps (at max 40KB/s) Few clients will keep longer connections uploading more, but a majority only sync under 100MB. Thank you for any help, I appreciate it! --Gian G. Spicuzza -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync Digest, Vol 37, Issue 10
A vanilla RHEL3 install or a RHEL3 install with a vanilla kernel? Just making sure here... --Gian On Sat, Jan 07, 2006 at 10:22:57AM -0500, Tim Boyer wrote: It's not many files, but since I'm running as root, I'm curious why there's a problem creating/deleting/unlinking these files. Only your OS can tell you for sure, but things to check for include: accessing networked filesystem data that removes foreign root privs (making root's access happen as "nobody"), and protections that are not UID/GID-based, such as SELinux's ACLs. ..wayne.. It's vanilla RHEL3, so no ACLs. It's local filesystems only, so there _shouldn't_ be the 'foreign root' problem. Thanks; I'll keep looking... ___ Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html rsync mailing list rsync@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync rs -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: ssh protocol 2
Do a google search for "cwrsync". -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
