On Fri, 2004-12-03 19:09:11 -0800, Brandon Knitter <[EMAIL PROTECTED]> wrote in message <[EMAIL PROTECTED]>: > 1) What strategies do other people employ?
Some time ago, I had a similar problem; basically, a little number of application worked on some kind of master repository, that needed to be synced to other locations (so it's a strict one-way sync). Since rsync'ing was too slow for me (thus, the remote sites were possibly too long running with inconsistent data, too long times between two rsync runs, ...), I took another approach. I wrote a little lib (loaded through LD_PRELOAD=... in front of the data-generating applications) that intercepted open(), close(), read(), write(), ... and replicated it over the network to the other sites. This way, I only needed to stop the system once, replicate it, and re-start the "master" side. However, this only works if you've got *very* reliable hardware: a network outage etc. would block the master servers. Unfortunately, this code isn't available freely by any means, but the idea is there. Maybe I'd rewrite it at some time, since it prooved to be quite useful. As an addition, maybe file-checking could be done in advance (size, time stamps) so that stopping the whole system before accepting a remote site could be avoided... > 2) Is there a better tool out there for moving large amounts of files? Not really. IFF you can actually use rsync, it's a really cool tool. I use it eg. for keeping Debian repos in sync and it just works (TM). Doing a more timely remote replication isn't rsync's task, so it isn't a good fit for this. > 3) Is there a way to save the filelist to a file instead of in memory? Even if you'd press the file list to a file: you'd need to mmap() it, in parts (4GB limit on 32bit hardware, which you aren't even allowed to fully exploit...), which would complicate things, and make it possibly a lot slower. MfG, JBG PS: Thanks to all who wrote code for rsync -- I'm really a happy user! -- Jan-Benedict Glaw [EMAIL PROTECTED] . +49-172-7608481 _ O _ "Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O fuer einen Freien Staat voll Freier BÃrger" | im Internet! | im Irak! O O O ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
signature.asc
Description: Digital signature
-- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
