Hi folks,
During an HPC talk some years ago, I recall someone mentioned a tool which can copy large datasets across a cluster using a ring topology. Perhaps someone here knows of this tool?
More to the point, we are pushing around datasets that are about 1Gbyte. The datasets are pushed out to dozens of nodes all at once and we foresee saturating the I/O system on our cluster as we grow. We are limited to using just the available disks and are looking for a reasonable solution that can support this kind of simultaneous access. Currently we push the data out using rsync, but if I don't get any better ideas I may simply move to a pull system where the data is fetched by HTTP. I can get better throttling that way, at least.
-geoff Geoff Galitz [EMAIL PROTECTED]
_______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
