Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Giovanni Tirloni
On Wed, May 4, 2011 at 9:04 PM, Brandon High bh...@freaks.com wrote: On Wed, May 4, 2011 at 2:25 PM, Giovanni Tirloni gtirl...@sysdroid.com wrote: The problem we've started seeing is that a zfs send -i is taking hours to send a very small amount of data (eg. 20GB in 6 hours) while a zfs

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Paul Kraus
On Thu, May 5, 2011 at 2:17 PM, Giovanni Tirloni gtirl...@sysdroid.com wrote: What I find it curious is that it only happens with incrementals. Full send's go as fast as possible (monitored with mbuffer). I was just wondering if other people have seen it, if there is a bug (b111 is quite old),

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Brandon High
On Thu, May 5, 2011 at 11:17 AM, Giovanni Tirloni gtirl...@sysdroid.com wrote: What I find it curious is that it only happens with incrementals. Full send's go as fast as possible (monitored with mbuffer). I was just wondering if other people have seen it, if there is a bug (b111 is quite old),

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread David Dyer-Bennet
On Tue, May 3, 2011 19:39, Rich Teer wrote: I'm playing around with nearline backups using zfs send | zfs recv. A full backup made this way takes quite a lot of time, so I was wondering: after the initial copy, would using an incremental send (zfs send -i) make the process much quick because

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Edward Ned Harvey
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Rich Teer Also related to this is a performance question. My initial test involved copying a 50 MB zfs file system to a new disk, which took 2.5 minutes to complete. The strikes me as

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Edward Ned Harvey
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Rich Teer Not such a silly question. :-) The USB1 port was indeed the source of much of the bottleneck. The same 50 MB file system took only 8 seconds to copy when I plugged the drive

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Rich Teer
On Wed, 4 May 2011, Edward Ned Harvey wrote: I suspect you're using a junky 1G slow-as-dirt usb thumb drive. Nope--unless an IOMega Prestige Desktop Hard Drive (containing an Hitachi 7200K RPM hard drive with 32MB of cache) counts as a slow as dirt USB thumb drive! -- Rich Teer, Publisher

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Rich Teer
On Wed, 4 May 2011, Edward Ned Harvey wrote: 4G is also lightweight, unless you're not doing much of anything. No dedup, no L2ARC, just simple pushing bits around. No services running... Just ssh Yep, that's right. This is a repurposed workstation for use in my home network. I don't

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Giovanni Tirloni
On Tue, May 3, 2011 at 11:42 PM, Peter Jeremy peter.jer...@alcatel-lucent.com wrote: - Is the source pool heavily fragmented with lots of small files? Peter, We've some servers holding Xen VMs and the setup was create to have a default VM from where others would be cloned so the space

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Brandon High
On Wed, May 4, 2011 at 2:25 PM, Giovanni Tirloni gtirl...@sysdroid.com wrote:   The problem we've started seeing is that a zfs send -i is taking hours to send a very small amount of data (eg. 20GB in 6 hours) while a zfs send full transfer everything faster than the incremental (40-70MB/s).

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Randy Jones
On 05/03/11 22:45, Rich Teer wrote: True, but the SB1000 only supports 2GB of RAM IIRC! I'll soon be Actually you can get up to 16GB ram in a SB1000 (or SB2000). The 4GB dimms are most likely not too common however the 1GB and 2GB dimms seem to be common. At one time Dataram and maybe

[zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Rich Teer
Hi all, I'm playing around with nearline backups using zfs send | zfs recv. A full backup made this way takes quite a lot of time, so I was wondering: after the initial copy, would using an incremental send (zfs send -i) make the process much quick because only the stuff that had changed between

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Eric D. Mudama
On Tue, May 3 at 17:39, Rich Teer wrote: Hi all, I'm playing around with nearline backups using zfs send | zfs recv. A full backup made this way takes quite a lot of time, so I was wondering: after the initial copy, would using an incremental send (zfs send -i) make the process much quick

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Peter Jeremy
On 2011-May-04 08:39:39 +0800, Rich Teer rich.t...@rite-group.com wrote: Also related to this is a performance question. My initial test involved copying a 50 MB zfs file system to a new disk, which took 2.5 minutes to complete. The strikes me as being a bit high for a mere 50 MB; are my

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Rich Teer
On Wed, 4 May 2011, Peter Jeremy wrote: Possibilities I can think of: - Do you have lots of snapshots? There's an overhead of a second or so for each snapshot to be sent. - Is the source pool heavily fragmented with lots of small files? Nope, and I don't think so. Hopefully a silly