Hard links revisited
I can't seem to get the hardlinks to work in windows via --link-dest. It just refuses to find the files in the directory and make a hard link. Here's the setup. -I run 2 rsyncs from batch files called sync.bat and sync2.bat. -I'm connected to the server via a mapped network drive, we'll call it 'z'. -I then run the client, on the destination, once to pull the data from the server and make a base-level mirror in a directory I call "source". -Next, I want to run the backup again and put it in a different directory called "mirror", but only hardlink the files that already exist in my base-level mirror. Every time that I run the first backup to "source", everything works fine and all of the files are moved to the location desired. Then when I run the second backup, which is supposed to hardlink to the first one if the file already exists, it merely copies all the files and doesn't make a single hardlink. //References sync.bat, first backup (z is the mapped network drive,destination is #the machine I am on) "c:\program files\cwrsync\rsync.exe" -avr --stats "/cygdrive/z/users/scott's folder/testBackup/" destination::testBigCopy #sync2.bat second backup - should use hardlinks "c:\program files\cwrsync\rsync.exe" -avr --stats --link-dest="/cygdrive/c/source" "/cygdrive/z/Users/Scott's Folder/testBackup/" destination::testHardLinks #rsyncd.conf [testBigCopy] path = /cygdrive/c/source/ read only = false transfer logging = yes [testHardLinks] path = /cygdrive/c/Mirror/testBackup/ read only = false transfer logging = yes Any ideas? I've tried nearly everything that I could think of that had even the slightest modicum of validity for a resolution. Nothing works yet. -- "Of course, that's just my opinion; I could be wrong" -Dennis Miller -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
@ERROR: auth failed on module
Hi, My nightly rsync jobs recently began to fail. I don't recall making any modifications to the rsync config files or upgrading rsync. Below is the error I receive when I try to rsync the home directory of serverA to serverB. Is there anything obviously wrong? Thanks. serverA rsync # rsync -arczv --delete --password-file=/etc/rsync/rsync.password /home 192.168.1.4::home @ERROR: auth failed on module home rsync: connection unexpectedly closed (88 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) /var/log/rsyncd.log on ServerB 2005/06/07 11:29:24 [22075] rsyncd version 2.6.0 starting, listening on port 873 2005/06/07 11:29:35 [20870] auth failed on module home from serverA.domain.com (192.168.1.3) Server Setup ---/etc/rsyncd/rsyncd.conf--- pid file = /var/run/rsyncd.pid log file = /var/log/rsyncd.log max connections = 2 [home] path = /home comment = /home on serverA server uid = root gid = root auth users = root hosts allow = 192.168.1.3 use chroot = yes read only = no list = yes secrets file = /etc/rsyncd/rsyncd.secrets strict modes = true ---/etc/rsyncd/rsyncd.secrets--- root:really_long_password_here Client Setup ---/etc/rsync/rsync.password--- really_long_password_here -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: @ERROR: auth failed on module
On Tue, Jun 07, 2005 at 12:07:49PM -0400, [EMAIL PROTECTED] wrote: > 2005/06/07 11:29:35 [20870] auth failed on module home from > serverA.domain.com (192.168.1.3) If you upgrade the server to 2.6.5, rsync will log the reason that the auth failed on that log-file line (password mismatch, missing secret, unauthorized user, invalid challenge response) Also, it doesn't have the bug (fixed in 2.6.3) that causes a secret (or password) file's last (or only) line to be ignored if it doesn't end in a newline (which you might also check for). If you can't upgrade, try various failure points, like manually specifying the rsyncd username ([EMAIL PROTECTED]::module), inputting the password manually, etc., to try and narrow down the failure. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Hard links revisited
On Tue, Jun 07, 2005 at 10:51:19AM -0400, Scott Lovenberg wrote: > #sync2.bat second backup - should use hardlinks > "c:\program files\cwrsync\rsync.exe" -avr --stats > --link-dest="/cygdrive/c/source" "/cygdrive/z/Users/Scott's > Folder/testBackup/" destination::testHardLinks For that to work the subdirectory cygdrive/c/source would need to exist on the destination *inside* the testHardLinks module's dir. When using an rsync daemon, no files may be referenced from outside of a module's path. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2758] "File exists" error using options -b and --backup-dir with device files.
https://bugzilla.samba.org/show_bug.cgi?id=2758 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |ASSIGNED --- Additional Comments From [EMAIL PROTECTED] 2005-06-07 10:49 --- Yes, this particular problem started in 2.6.4 because prior to that, rsync was failing to backup any updated devices (and apparently symlinks as well). A quick fix would be to try to delete the item in the way in the backup dir, but the backup code is systemically bad at overwriting existing objects (such as replacing a file with a dir or visa versa), so the code really needs some deep surgery. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2758] "File exists" error using options -b and --backup-dir with device files.
https://bugzilla.samba.org/show_bug.cgi?id=2758 --- Additional Comments From [EMAIL PROTECTED] 2005-06-07 10:56 --- Created an attachment (id=1264) --> (https://bugzilla.samba.org/attachment.cgi?id=1264&action=view) A partial fix My version of the quick-fix that only deals with removing a device/file/symlink that's in the way of a device or symlink. It also fixes a couple problems in the logic in the code for backing up a device or a symlink. I didn't see any reason to constrain the prior object backed up to being another device when backing up a device, so my code just tries to get rid of anything that's in the way. It will still fail for dir/non-dir mismatches, but that's a deeper problem that has been around for a very long time. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: @ERROR: auth failed on module
On Tue, 7 Jun 2005, <[EMAIL PROTECTED]> wrote: Thanks for the detail in your report. Error messages, log messages, config files all in one email. :) > /var/log/rsyncd.log on ServerB > > 2005/06/07 11:29:24 [22075] rsyncd version 2.6.0 starting, listening on port > 873 > 2005/06/07 11:29:35 [20870] auth failed on module home from > serverA.domain.com (192.168.1.3) I don't know if the fork() happens before or after the auth, but I would expect the PID values to be the same or the child's PID to be slightly higher after only 11 seconds. So the PID values imply that you have multiple daemons running. Have you checked for that? John -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: AIX rsync backup problem -- cp with hard link fails
On Thu, Jun 02, 2005 at 11:34:45PM -0400, Sherry Sun wrote: > "Cannot link to a file on another device." > How should I get around this problem? Your only choices are to turn off -H (so that rsync copies hard-linked files as separate files) or to ensure that all dirs containing files that are cross-linked are on the same filesystem (perhaps by adjusting your destination hierarchies, or perhaps by using symlinked destination dirs and the --keep-dirlinks option to rsync). ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Cannot Allocate Memory
Hello - I've just upgraded to 2.6.5. On a new set of servers I'm trying to sync I seem to be running into a log of memory allocation errors. With some of the servers, I start the rsync session, and it counts so many files and then stops. At that point I'm unable to control-c out of the process. From another session I can't issue a kill command either...very strange. On other servers, the first few files/directories issue a 'cannot allocate memory', but strangly enough, it continues on. Does this simply mean that the 256mb memory isn't sufficient? I am trying to sync a very large folder structure. Is there anyway around this using tmp disk space or something until I can get more memory? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to --exclude /usr/ but --include /usr/local/ ?
Wayne Davison wrote: On Sun, Jun 05, 2005 at 11:28:08PM -0500, Larry Alkoff wrote: In looking at man rsync, it does not appear to both --exclude /usr/ and --include /usr/local/ Correct, you can't exclude /usr and still get to /usr/local. What you want to do is to include /usr/local/ and exclude /usr/local/* (in that order) without specifying a rule for /usr (unless you need to override a more general exclude). I start rsync at / and the files I wish to save are in /usr/local plus /home and some other directories. I'd suggest putting +/- rules into a file and using --exclude-from. For instance: + /home/ + /usr/ + /usr/local/ - /usr/local/* + /some/ + /other/ + /other/directories/ - /other/* - /* ..wayne.. Thank you very much _again_ for very informative reply to my question. In my case I wanted to start at / and include directories of interest to me for backup purposes. /usr is not needed but /usr/local is. I solved the problem another way some months ago using a for loop which selects the directories I want to save. All except /usr/local start from / so I thought it might be possible to simplify the script. Guess I'll stick with the for loop which works well. Larry -- Larry Alkoff N2LA - Austin TX Using Thunderbird on Slackware Linux -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Behavior of --checksum, suggestion
Hi rsync folks, I understand that by default, rsync will re-copy a file if the timestamp differs, but the size is the same. Sometimes, though, I have large files whose mtime have been changed but are otherwise identical. It seems a solution to this is --checksum, but it seems like --checksum computes the checksum for ALL files, which is time consuming. What would be nice is if we could have a --checksum=ifneeded, which would compute the checksum of two files only when their size is the same but the mtime differs. If the files turned out to have the same checksum, the destination's mtime would simply be updated without the whole file being re-transferred. Is this something that would be hard to implement? Thanks, Moshe -- Moshe Jacobson -- http://runslinux.net -- AIM: Jehsom | This message is strictly confidential; it should be shown to nobody | | except the listed addressee(s). If you've received this message in | | error, please promptly delete it and notify the sender. | -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Behavior of --checksum, suggestion
On Tue, Jun 07, 2005 at 04:46:43PM -0400, Moshe Jacobson wrote: > Is this something that would be hard to implement? Yes, it would be rather awkward in the current protocol. The way things currently work, the receiving side needs to have all relevant info already available in order to decide what files to send. One possible solution would be to add an extra "phase" prior to the file-transfer phase (before the receiving side forks) where the receiver makes a pass through the files and requests checksums for files that might need to be transferred, and then proceeds on to the normal checksum-sending phase (after forking) but with an optimization that allows it to avoid calling stat() on all the files all over again (it would flag changed files, required dirs, etc. during the earlier checksum phase). So, that would be quite a big change, which probably means that it won't happen -- at least, not in the current rsync (hopefully a future, improved-protocol rsync will be more flexible in this regard, and it would sure be nice to get around to working on it again before too long). ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Behavior of --checksum, suggestion
Thanks for the prompt reply, Wayne. I just did a test and found that --partial will prevent rsync from transferring the entire file again when the mtimes vary but the sizes are the same. I suppose this is sufficient, but how does it determine that the files are the same if it's not doing a checksum or transferring the entire file? Moshe whole file again if the size is the same and the mtime is different On Tue, 7 Jun 2005, Wayne Davison wrote: On Tue, Jun 07, 2005 at 04:46:43PM -0400, Moshe Jacobson wrote: Is this something that would be hard to implement? Yes, it would be rather awkward in the current protocol. The way things currently work, the receiving side needs to have all relevant info already available in order to decide what files to send. One possible solution would be to add an extra "phase" prior to the file-transfer phase (before the receiving side forks) where the receiver makes a pass through the files and requests checksums for files that might need to be transferred, and then proceeds on to the normal checksum-sending phase (after forking) but with an optimization that allows it to avoid calling stat() on all the files all over again (it would flag changed files, required dirs, etc. during the earlier checksum phase). So, that would be quite a big change, which probably means that it won't happen -- at least, not in the current rsync (hopefully a future, improved-protocol rsync will be more flexible in this regard, and it would sure be nice to get around to working on it again before too long). ..wayne.. -- Moshe Jacobson -- http://runslinux.net -- AIM: Jehsom | This message is strictly confidential; it should be shown to nobody | | except the listed addressee(s). If you've received this message in | | error, please promptly delete it and notify the sender. | -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Behavior of --checksum, suggestion
On Tue, Jun 07, 2005 at 05:44:25PM -0400, Moshe Jacobson wrote: > I just did a test and found that --partial will prevent rsync from > transferring the entire file again when the mtimes vary but the sizes > are the same. No, --partial tells rsync to keep a partially transferred file. Rsync will always do an efficient update up a file unless --whole-file is specified (or implied by a local transfer). This means that the generator sends some per-block checksum data to the sender, and the sender sends some block-matching commands to the receiver, and the receiver copies the file locally with no actual file data going over the wire (when the files were actually identical in contents, but not in mtime). ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: @ERROR: auth failed on module
John & Wayne, :) I'm fairly certain no other rsync processes were running, a reboot was part of the troubleshooting process, as was a 'killall rsync' or two. I did try specifying the user ([EMAIL PROTECTED]::home), and manually gave the password. I did not have the newline character at the end of the secrets & password files, but did add them to no avail. Upgrading to 2.6.5 would be possible, but I worked around the issue by going anonymous. These two servers are connected via a crossover cable, and rsyncd is only listening on the ip of that interface, so it's not a security issue. I'd still like to find out what caused it to seemingly stop working for no reason, the last successful rsync occured on June 1, and rsync hadn't been updated since April 15. The only package that was updated on June 1 was freeradius, which I can't see causing this. We did have a UPS failure, but that was 5 days prior to last successful rsync job, so doubt that caused it either. I'm stumped. Thanks for the help guys! Thomas John Van Essen wrote: On Tue, 7 Jun 2005, <[EMAIL PROTECTED]> wrote: Thanks for the detail in your report. Error messages, log messages, config files all in one email. :) /var/log/rsyncd.log on ServerB 2005/06/07 11:29:24 [22075] rsyncd version 2.6.0 starting, listening on port 873 2005/06/07 11:29:35 [20870] auth failed on module home from serverA.domain.com (192.168.1.3) I don't know if the fork() happens before or after the auth, but I would expect the PID values to be the same or the child's PID to be slightly higher after only 11 seconds. So the PID values imply that you have multiple daemons running. Have you checked for that? John -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
BUG REPORT: latest rsync misinvokes ssh with [EMAIL PROTECTED] ...
The latest rsync (2.6.5) seems to invoke the transport specified by
$RSYNC_RSH differently. I have this set to point at an ssh wrapper script
(which is now I noticed), and an strace shows this:
[archives/[EMAIL PROTECTED]> strace -e trace=process -f /opt/bin/rsync -avHP
/mnt/phat/archives/silva/. [EMAIL PROTECTED]:/a/ali/home/.
[...snip...]
execve("/u/cameron/scripts/rsync_rsh", ["rsync_rsh", "ali", "-l", "root",
"rsync", "--server", "-vlHogDtpr", "--partial", ".",
"/a/ali/home/."...], [/* 201 vars */]) = 0
in short, rsync is seeing the "[EMAIL PROTECTED]" and invoking:
ssh ali -l root rsync-daemon-invocation...
This is Very Wrong. It should be:
ssh -l root ali rsync-daemon-invocation...
or better still just:
ssh [EMAIL PROTECTED] rsync-daemon-invocation...
i.e. not have rsync try to be smart about the host: prefix to the target.
I've hacked my wrapper to cope with this misinvocation, but even a
transient glance at the ssh manual entry SYNOPSIS section should show
this is wrong, and of course in violation of all the usual UNIX command
line conventions where the options preceed the arguments.
I can only assume some loose parsing in the ssh command itself has
permitted this to go unnoticed.
Personally I'm of the opinion that rsync shouldn't parse the
"[EMAIL PROTECTED]:" prefix at all (other than detecting it), and should not
break it into "-l user" and "host".
Can someone elucidate the reasoning behind this change?
Cheers,
--
Cameron Simpson <[EMAIL PROTECTED]> DoD#743
http://www.cskk.ezoshosting.com/cs/
If you pick up a starving dog and make him prosperous, he will not bite you.
This is the principal difference between a dog and a man. - Mark Twain
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
