On Fri, Jul 23, 2010 at 6:29 PM, Tom Christensen <pav...@live.com> wrote:
> > > > Date: Fri, 23 Jul 2010 10:02:26 +0200 > > From: m...@citd.de > > > To: pav...@live.com > > CC: rsync@lists.samba.org > > Subject: Re: rsync to iSCSI over WAN > > > > On 23.07.2010 00:30, Tom Christensen wrote: > > > > > > I am running rsync in cygwin on windows. I am attempting to backup a > > > somewhat large data store (750GB) to a remote site. As its windows > > > and preserving permissions exactly is important, I have an iSCSI drive > > > mounted on the local system across a somewhat slow WAN link (IE, it > > > would take about 3 months to copy the datastore over it). > > > Unfortunately, since this appears as a "local" copy to rsync, it > > > always copies whole files. Even though it is a "local" copy, I want > > > to only send diffs, as we have large files that have small changes > > > daily. Reading the man page, and everything I can find on the net I > > > don't see an option to force diffs only/rsync protocol, is this > > > possible? > > > > You have an abstraction error here, an iSCSI device is just a bit-bucket > > like any localy connected HDD in that all filesystem and data processing > > is done on the local side and raw block-data is send over the WAN. You > > would have to run the other side of rsync on the machine that provides > > the iSCSI-Device. For that to work the remote-machine would have to > > mount the filesystem localy which in most cases means you would have to > > unmount it from your Windows machine. This is because, except for > > cluster-filesystems or a filesystem that is mounted read-only (on every > > machine!), a given fileystem can only be mounted on 1 machine at a time. > > > > > > > > > > > > > > Bis denn > > > > -- > > Real Programmers consider "what you see is what you get" to be just as > > bad a concept in Text Editors as it is in women. No, the Real Programmer > > wants a "you asked for it, you got it" text editor -- complicated, > > cryptic, powerful, unforgiving, dangerous. > > > > I am aware of that fact, it is precisely the fact that the filesystem is > exposed directly to windows that I need iSCSI. Mounting an ext3 partition > shared via smb/cifs (another option on this NAS device) does not provide me > with permissions fidelity (as ext3 does not support all of the ntfs > permissions. I was under the impression that rsync would calculate the > diffs between the "local" copy and the "remote" (iSCSI) copy, and then only > "send" the diffs, IE only the diffs would be written to disk (and therefore > only the diffs would be sent via block-io over the WAN to the iSCSI). But > that very well could be a misunderstanding on my part of the way rsync > functions at that low level. Maybe on the remote side it reads through the > whole file, writing it out to disk and inserting the diffs it receives? In > which case no matter what the whole file would be sent over block-io? This > functionality would also mean that even at the "network share" level > (smb/cifs) at the remote side, you would gain nothing from --no-whole-file > because the remote side would do the same thing IE write the whole file to > the share anew resulting in the same amount of network traffic. > > At any rate, --no-whole-file appeared to improve performance on the backup > last night, running the backup with --no-whole-file resulted in about 75% > less data copied (according to the report at the end) and the backup ran in > about half the time it was taking previously. Granted it could have just > been a slow day in the office and maybe not that many files were changed... > a sample size of 1 is not representative.. > > What kind of system is hosting the remote iSCSI? If you need NTFS, then you could set up a iSCSI drive on on top of ZFS/FreeBSD - locally. Then setup a similar system remotely. Do your backups against the local system NTFS-NTFS, then send deltas/snapshot diffs to the remote system over WAN (using rsync or "zfs send") k
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html