Re: another iconv question
On Mar 31, 2008, at 10:14 PM, Wayne Davison wrote: On Mon, Mar 31, 2008 at 09:54:16AM -0400, Robert DuToit wrote: we tried adding option iconv=C and iconv=C,C but no luck oddly this is what was returned when he ran "locale" on his machine-it didn't specify a locale or charset: xserve-backup-02:/Volumes/Backup RAID 8TB teleclub$ locale LANG= LC_COLLATE="C" LC_CTYPE="C" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL="C" but I found this for German-Swiss in my "locale -a" list: de_CH.ISO8859-1 de_CH.ISO8859-15 de_CH.UTF-8 so we are not sure which charset to use with iconv=. ISO8859-1 or ISO8859-15 or UTF-8? That would appear to be a request for rsync to not do any character conversion, in which case you shouldn't be using --iconv. When using the --iconv option, you should describe the character set that the files are in on the disk on both the sender and the receiver. [rsync] re-copies [e]very file with an umlaute. Are you using (or implying) the -t (--times) option? (Use the -i option to see if the copy is due to time changes or something else.) What do you mean by "re-copies"? when he runs rsync again, it creates a new copy of the files (with umlautes) even though they aren't changed. He said the mod times are the same on sender and receiver. Are you talking about copying the file away and back again, which creates an extra copy of the file on the source machine under a slightly different name? If so, perhaps that character cannot be converted to UTF-8 and back without changing in the process. If you're talking about a single direction, make sure you've specified the right character set for each machine. Also make sure that the file wasn't copied before with the wrong character set, since the new file might be the right one. Yeah- he ran it many times and it always copies over the umlaute files. I'll experiment with the de_CH charsets above though. Thanks, Rob ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe
On Mon, Mar 31, 2008 at 10:48:18AM +0200, Sebastian Kösters wrote: > On the sun the command (in ps ef) went away after about 2 Minutes with > the error message I sent you. That makes it look like a connection failure then. Perhaps a firewall is closing an idle connection? e.g. I once had a Linksys box that would close my ssh connections for me unless I set a keep-alive option in the protocol. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: another iconv question
On Mon, Mar 31, 2008 at 09:54:16AM -0400, Robert DuToit wrote: > we tried adding option iconv=C and iconv=C,C but no luck That would appear to be a request for rsync to not do any character conversion, in which case you shouldn't be using --iconv. When using the --iconv option, you should describe the character set that the files are in on the disk on both the sender and the receiver. > [rsync] re-copies [e]very file with an umlaute. Are you using (or implying) the -t (--times) option? (Use the -i option to see if the copy is due to time changes or something else.) What do you mean by "re-copies"? Are you talking about copying the file away and back again, which creates an extra copy of the file on the source machine under a slightly different name? If so, perhaps that character cannot be converted to UTF-8 and back without changing in the process. If you're talking about a single direction, make sure you've specified the right character set for each machine. Also make sure that the file wasn't copied before with the wrong character set, since the new file might be the right one. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: using rsync on raw device
On Sun, Mar 30, 2008 at 03:20:59PM -0400, Matt McCutchen wrote: >On Sun, 2008-03-30 at 15:00 -0400, George Georgalis wrote: >> I'm trying to use rsync to manage a raw disk image file. >> >> rsync --checksum --perms --owner --group --sparse --partial --progress \ >> 192.168.80.189:/dev/rwd0d /u0510a/rwd0d.img >> skipping non-regular file "rwd0d" > >Use the --copy-devices option added by this patch: > >http://rsync.samba.org/ftp/rsync/patches/copy-devices.diff hey this looks like a great patch, Thanks! I don't have an environment to test it atm but will give it a whirl when I do. // George -- George Georgalis, information system scientist < -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync 3.0.1pre3 released
On Mon 31 Mar 2008, Wayne Davison wrote: > Hopefully this will be the last pre-release of 3.0.1 before the final > release. Please test this and send email to the rsync mailing list with > any questions, comments, or bug reports. I've uploaded this to 'experimental', it should appear on http://packages.debian.org/rsync?suite=experimental&keywords=rsync in due course. That should be installable on Debian "testing". Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: bwlimit on rsync locally
Thanks John. I looked into this patch but it doesn't resolve my problem, and we have have posix FADV. I found a simple solution. Just open the file on receiver to be write direct to disk avoid file buffer/cache. Of course the performance sucks but it's what I need. The output block rate cannot be reduced by 2/3 based on my besting on linux 2.6.22. I can post the diff if anybody wants. - Reeve On Sun, Mar 30, 2008 at 10:23 PM, John Van Essen <[EMAIL PROTECTED]> wrote: > On Fri, 28 Mar 2008, Reeve Yang <[EMAIL PROTECTED]> wrote: > > Is there anyway to make rsync use direct I/O bypassing FS buffer? > > I'm not an expert on that by any means, but AFAIK, rsync supports that > only via this patch: > > http://rsync.samba.org/ftp/rsync/patches/drop-cache.diff > > (You'll have to compile your own version of rsync.) > > > > John > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
another iconv question
Hi All, I am trying to help my friend set up his rsync with iconv. Presently it works fine but re-copies every file with an umlaute in the filename. I saw a recent post about this and the fix but... he ran "locale" ( both source and dest are on same machine) and is running on a German-Swiss locale xserve-backup-02:/Volumes/Backup RAID 8TB teleclub$ locale LANG= LC_COLLATE="C" LC_CTYPE="C" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL="C" we tried adding option iconv=C and iconv=C,C but no luck - it still re-copies very file with an umlaute. I checked "iconv --list" on my Mac and see no "C" listed. I am not sure if "C" is correct either. Example: The correct file name would be "Action des Monats für vertonung.mov" and not "Action des Monats f\#303\#274r vertonung.mov" but the log shows it not translated: /Volumes/SAN_Video/Final Cut Pro Documents/Capture Scratch/Action des Monats/! Render/Action des Monats f\#303\#274r vertonung.mov 32768 0% 344.09kB/s0:54:11 42205184 3% 40.25MB/s0:00:26 Though the actual file name gets copied correctly to dest, obviously the mapping (if that is what it is called) is different causing rsync to update the file every time. Any thoughts or direction appreciated. Thanks, Rob D -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Windows backup API in rsync port?
Tomasz Chmielewski schrieb: Windows systems have a great "feature" - even administrators can't access files for which they don't have permissions (I'm not talking about files exclusively opened, which is something else). To read files which are normally inaccessible because of insufficient permissions, Windows administrator can use Windows backup API. Unfortunately, rsync Windows port doesn't seem to use this feature. There was a similar thread about it 4 years ago - did anything change in rsync from this time? http://lists.samba.org/archive/rsync/2004-February/thread.html#8677 I asked at Cygwin mailing list, and I'm glad to know that this issue is fixed in the recent snapshots of Cygwin: just replace your cygwin1.dll library with a recent snapshot from http://cygwin.com/snapshots/ and rsync on Windows will be able to open the files where Administrator is not allowed to. See the whole thread here: http://cygwin.com/ml/cygwin/2008-03/threads.html#00662 Great! -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: replicating a live directory
On Sun 30 Mar 2008, Joao Miguel Ferreira wrote: > > An example: if we do not stop sendmail were rsync'ing can we be sure that > the queues on the replica machine are in a consisten state ? I don't know how sendmail organizes things, but at least with exim there aren't any extra indexes or such that need to be consistent with the spool. > same for dhcpd leases or squid's cache ? dhcpd leases should not be a problem. I'd not even consider syncing a squid cache. After all, it's a cache, not important data... Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RFE: Lockfile option for use in cronjobs
Hi, Looking for an easy way to prevent a repetitive rsync to be running multiple times, I was wondering if it could be useful to have an option like: --pidfile /some/path/rsync-mirror-org.pid So that rsync can be run directly from cron without requiring a wrapper script to do pidfile handling. This way rsync on startup could check the pid-file, see if another rsync is using this pid, and bail out with an error if it is. Otherwise clean up the stale pidfile and continue. I think this would be very useful to instruct mirrors how to configure it, rather than providing some script that needs local customizations. Thanks in advance, -- -- dag wieers, [EMAIL PROTECTED], http://dag.wieers.com/ -- [Any errors in spelling, tact or fact are transmission errors] -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hanging at "generate_files phase=1"
On Fri, 21 Mar 2008 11:45:20 +0100 Daniel Maher <[EMAIL PROTECTED]> wrote: > I am experiencing a (consistent, reproducible) condition whereby rsync > is hanging indefinitely at the following stage : > generate_files phase=1 > > The condition does not occur if --dry-run is specified - only when the > run is "for real". Finally, i can scp the files without problems, so > it is not a general connection issue, nor is it a problem with > permissions (everything is running via sudo in any case). In case anybody is interested, here's my follow-up to this issue : Eschewing the 2.6.9 that ships by default on Cygwin, I compiled Matt's patched rsync 3.0.0 by hand and moved forward with that binary. After trying a few more scenarios - and hanging each time at the same phase noted above - i tried the --whole-file switch (copy files whole), which solved the problem nicely. Now, of course, i'm not sure /why/ that worked (which is to say that i'm not sure why the algorithm was failing), but there you go. :) -- Daniel Maher signature.asc Description: PGP signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
AW: rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe
I do not think that this is the Problem. Both Server are reachable from each other without Problems. When I start the ssh/rsync command on the Backuppc you can see the command via ps ax all the time. It never stops with a timeout or something like that. On the sun the command (in ps ef) went away after about 2 Minutes with the error message I sent you. So Backuppc is thinking that the rsync command is running but it has stopped a long time before. I have 2 suns. One with rsync 2.6.9 and one with 3.0.0. Both have the Problem. I also have 40 other Server (most of them linux) and the same command thats not working on these 2 suns is working on the other machines (also on 3 other suns with the same version I am using but I dont find what is different between this Servers - they have also the same ssh version). Regards Sebastian -Ursprüngliche Nachricht- Von: Wayne Davison [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 28. März 2008 17:03 An: Sebastian Kösters Cc: rsync@lists.samba.org Betreff: Re: rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe On Fri, Mar 28, 2008 at 04:41:26PM +0100, Sebastian Kösters wrote: > 2008/03/28 16:35:46 [5573] rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32) This tells you that the connection unexpectedly closed, so you should look to see why BackupPC went away. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync 3.0.1pre3 released
I have released rsync 3.0.1pre3 for testing. This version contains a couple more bug fixes since the 3.0.1pre2 release: - Don't allow a daemon-exclude of .* or */ to affect a (vital) "." dir. - Fixed a glitch in the path-checking of the daemon-exclude code that could check a relative arg against the wrong current directory. - A couple minor chdir() improvements in the arg-parsing code. - Improved the hard-link testsuite script. Hopefully this will be the last pre-release of 3.0.1 before the final release. Please test this and send email to the rsync mailing list with any questions, comments, or bug reports. To see a full summary of the changes since 3.0.0, visit this link: http://rsync.samba.org/ftp/rsync/src-previews/rsync-3.0.1pre3-NEWS You can download the source tar file and its signature from here: http://rsync.samba.org/ftp/rsync/src-previews/rsync-3.0.1pre3.tar.gz http://rsync.samba.org/ftp/rsync/src-previews/rsync-3.0.1pre3.tar.gz.asc The patches directory is now in a separate tar file (for those that want it): http://rsync.samba.org/ftp/rsync/src-previews/rsync-patches-3.0.1pre3.tar.gz http://rsync.samba.org/ftp/rsync/src-previews/rsync-patches-3.0.1pre3.tar.gz.asc ..wayne.. signature.asc Description: Digital signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
