On Wed, Jul 23, 2014 at 01:27:47PM +0200, Saso Kiselkov via illumos-discuss wrote:
> For async replication, incremental snapshots are the way to go. Just > transfer an incremental snapshot every 30-60s. They're relatively > inexpensive and robust, so pool corruption doesn't immediately get > propagated to the replication pair. We were looking to something like this recently, but ran into scalability issues with large filesystem counts. It took something like 5 minutes to do a zfs recv of 1000 filesystems (with no changes since last snapshot). The zfs send took only 16 seconds, crossing the network took seconds, but the recv performance broke it horribly. One of my colleagues that was working on it determined it was setting properties that was killing it, so rather that doing a send -R of everything everytime, we tried just sending filesystems that had changed. But while a send -R of 1000 filesystems took 16 seconds, generating individual send streams separately for 100 took 3 minutes 8-/. We ended up having to give up on the idea of near real time replication and go another way. ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
