Paul Slootman wrote: > On Sat 19 Jan 2008, Brian wrote: > >> I guess this is more of an rsync question, but does involve Dirvish too. > > You're right, it is more of an rsync question :) > >> Dirvish keeps around 15 copies of this system (#2) around, of course >> using Hard links as expected. >> >> The other Debian NSLU (#2) has a big hard disk, and every once in a >> while I rsync the vaults dirs of the NSLU (#1) to NSLU (#2) so as to >> give me a second copy of the backups. > > This means you're transferring a list that contains 15 copies of an > image (assuming there's only one image on #1). Doing this while > preserving all hard links is quite memory-intensive, when using current > versions of rsync. > >> Apart from splitting the vault into smaller parts of the system, any > > I'd typically do it by first transferring the first image, and then > transferring the second one in a similar way that dirvish does, i.e. by > using --link-dest that points to the first image, etc. > >> ideas how to reduce the memory overhead? Maybe its just a bad idea to >> use dirvish to backup the complete debian system? > > You'll always run into this sooner or later. Nothing wrong with > backuping a complete debian system. > > What you could try is the current prerelease version of rsync 3.0.0; > currently it's prerelease 8. I'm using that for a couple of systems, > and it's much better in doing large lists. For one, it doesn't wait > until the entire list is transferred before beginning with transferring > files. Doe note that you need that version on both ends for the new > protocol to work. > > > Paul Slootman > _______________________________________________ > Dirvish mailing list > [email protected] > http://www.dirvish.org/mailman/listinfo/dirvish > Paul, thanks. Just to be certain, the vault has about 15 trees in it, most of the copies are identical so mainly hard links, but I assume that would only make a difference to the amount of data that may need to be transferred, not to the size of the actual file list sent. So I'm not quite certain how --link-dest would help for my problem.
I guess the new version may actually make things worse for my situation, as transferring files at the same time would probably take even more storage. Unless it means that the file list in storage would be smaller at any one point in time. I think my main problem may just be the lousy paging performance of the NSLU2. Cheers Brian _______________________________________________ Dirvish mailing list [email protected] http://www.dirvish.org/mailman/listinfo/dirvish
