I was about to write something similar yesterday, but work interfered. ^o^
For bandwidth a RAID(Z*/6, don't even think about RAID5 or equivalent) is indeed very nice, but for IOPS it will be worse than a RAID10. Of course a controller with a large writeback cache can pretty alleviate or at least hide those issues up to a point. ^.^ On Wed, 28 May 2014 07:22:22 -0700 Scott Laird wrote: > IMHO, you were probably either benchmarking the wrong thing or had a > really unusual use profile. RAIDZ* always does full-stripe reads so it > can verify checksums, so even small reads hit all of the devices in the > vdev. That means that you get 0 parallelism on small reads, unlike most > other RAID5+ systems where disks can read independently. So, with an > 8-disk RAIDZ (RAIDZ2, RAIDZ3...) setup, you'll only get 1 disk worth of > read IOPS, instead of 6-8 disks worth. That's a pretty massive hit. > > The bandwidth, availability, and general hassle of RAIDZ2 is nice, > though. > > > On Tue, May 27, 2014 at 3:49 PM, Craig Lewis > <cle...@centraldesktop.com>wrote: > > > On 5/27/14 13:40 , phowell wrote: > > > > Hi > > > > First apologies if this is the wrong place to ask this question. > > > > We are running a small Ceph (0.79) cluster will about 12 osd's which > > are on top of a zfs raid 1+0 (for another discussion)... which were > > created on this version. > > > > > > Just a reminder to benchmark everything, especially things you have > > known to be true since the dawn of time. I benchmarked RAID10 vs. > > RAID5 so long ago, I had to find a 3.5" floppy to open the spreadsheet. > > > > > > Recently, I was testing ZFS on software encrypted volumes, and wanted > > to see how badly it would impact a PostgreSQL server. My test setup > > was using RAIDZ2, so I just ran the benchmark on that zpool. > > > > Imagine my surprise when an untuned and encrypted RAIDZ2 posted better > > benchmarks than a tuned ZFS RAID10. > > > > > > I really think the "RAID5 is bad for performance" is a nasty hold-over > > from when parity calculations needed dedicated hardware. I won't be > > building any more ZFS RAID10 arrays. > > > > > > -- > > > > *Craig Lewis* > > Senior Systems Engineer > > Office +1.714.602.1309 > > Email cle...@centraldesktop.com > > > > *Central Desktop. Work together in ways you never thought possible.* > > Connect with us Website <http://www.centraldesktop.com/> | > > Twitter<http://www.twitter.com/centraldesktop> | Facebook > > <http://www.facebook.com/CentralDesktop> | > > LinkedIn<http://www.linkedin.com/groups?gid=147417> | Blog > > <http://cdblog.centraldesktop.com/> > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > -- Christian Balzer Network/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
