> > Just how little an issue is "Data loss"? > If you truly only care about performance, then you just stripe across > disks with no redundancy at all. > But when the first disk dies, the pool will be broken.
Data loss isn't very important but we would like to be able to take a 1 disk drive hit to be able to replace it with a hot spare. Other than this performance is what matters. Have you thought about a 3-way mirror instead of RAID-Z? I started with > RAID-Z2 and went to a 3-disk mirror. Reads are faster, and I haven’t > noticed any degradation at all when writing vs the RAID-Z2 configuration I > used to have. Plus CPU time isn’t an issue for mirrors. CPU time isn't a big of an issue, but I think we will be choosing a mirror vdev setup instead RAIDZ because of the number of drives in the array. I assume you mean 'slog'. There's always a ZIL. Yes, I meant that - You do not require a separate SLOG (like zeusram) when the entire pool is set as "sync" disabled. Correct me if I am wrong. Your basic choices here are: > 6 x 4 TB disks, RAIDZ-2, single vdev. This will provide you with about > the same number of IOPS at a given latency as a single disk would > (probably somewhere between 60 and 100). > 8 x 4 TB disks, mirror, 4 vdevs. This will provide you with about 4x as > many IOPS at a given latency. We are aiming at getting as much IOPS as possible maintining our constraints (the concurrent 200 writes and 150 reads at 5mbit/s each, each read/write being sequential). We are planning on purchasing 12x 4TB disks. Using 1 or maximum of 2 as hot-spares. We are going to aim for a mirror setup, however we are unsure which setup is going to give the best performance for our use case. 10x4TB HDDs 5 vdevs (2 disks in each vdev). 9x4TB HDDs 3 vdevs (3disks in each vdev). 10x4TB HDDs in 2 vdevs (5 drives in each vdev). Which of the above mentioned configuration would be the best for our scenario? On Mon, Jul 7, 2014 at 6:33 PM, Keith Wesolowski < [email protected]> wrote: > On Mon, Jul 07, 2014 at 06:08:15PM +0200, Ibrahim Tachijian via > smartos-discuss wrote: > > > Hey Joyent and every other smartos user. > > > > We are trying to narrow down a few of our hardware choices to meet our > > requirement. > > > > Our requirements is well defined and we know exactly what we expect from > > the proposed array. > > We have barely any CPU power necessary so a single quad core is enough > for > > us. > > > > Requirements: > > > > > > - Write operations:* (24/7)* > > - *Approx 200 consecutive writes each write at 5mbit/s * > > - > > *[Total: 125MB/s writes] * > > - Read operations (*150 consecutive reads* is the UPPER limit, > might > > be less). > > - Approx 150 consecutive reads each read at* 5mbit/s* > > - [Total: *93MB/s reads*]. > > - Storage capacity 15TB of actual space (Not counting space lost to > > RAIDZ levels or mirrored vdevs) > > > > Our proposal is: > > > > > > - E5-1620v2 (Quad 3.7ghz Xeon) > > - 128gb of DDR3 ECC REG RAM > > - SAS Disks from HGST (Count and Size of each drive to be determined) > > - Proposed drive HGST SAS HUS724040ALS640 > > > > > > - ZFS "sync" setting set to disabled. Data loss is not an issue for > us, > > we only care about performance. > > - Dedupe off, compression off (We're saving video) > > - ZIL is unnecessary because of "sync" disabled. > > I assume you mean 'slog'. There's always a ZIL. > > > - RAIDZ level? We are unsure. > > > > Given these parameters, and the experience joyent has had with HGST 4TB > SAS > > drives. How large of an array, and what RAIDZ or mirrored vdev > > (raid10-like) would I require to be able to meet my above mentioned > > requirements? > > Your basic choices here are: > > 6 x 4 TB disks, RAIDZ-2, single vdev. This will provide you with about > the same number of IOPS at a given latency as a single disk would > (probably somewhere between 60 and 100). > > 8 x 4 TB disks, mirror, 4 vdevs. This will provide you with about 4x as > many IOPS at a given latency. > > Add a hot spare to either one if desired (I'd recommend it). Given the > modest difference in cost between the two options, I'd spring for the > mirror. If every dollar/euro counts or you have plenty of time for > testing, buy 6 and try RZ2 to start, measure performance, and if it's > not acceptable buy 2 more and rebuild the pool. The big question is > whether your reads and writes will be interleaved in such a way that the > disks can't stay in streaming mode; if so, you'll be very thankful for > the mirror. > -- Ibrahim Tachijian ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
