> -----Original Message----- > From: Garrett D'Amore [mailto:garr...@nexenta.com] > Sent: 15. juni 2010 17:43 > To: Arve Paalsrud > Cc: zfs-discuss@opensolaris.org > Subject: Re: [zfs-discuss] High-Performance ZFS (2000MB/s+) > > On Tue, 2010-06-15 at 04:42 -0700, Arve Paalsrud wrote: > > Hi, > > > > We are currently building a storage box based on OpenSolaris/Nexenta > using ZFS. > > Our hardware specifications are as follows: > > > > Quad AMD G34 12-core 2.3 GHz (~110 GHz) > > 10 Crucial RealSSD (6Gb/s) > > 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS expanders > > LSI2008SAS (two 4x ports) > > Mellanox InfiniBand 40 Gbit NICs > > Just recognize that those "NICs" are IB only. Solaris currently does > not support 10GbE using Mellanox products, even though other operating > systems do. (There are folks working on resolving this, but I think > we're still a couple months from seeing the results of that effort.) > > > 128 GB RAM > > > > This setup gives us about 40TB storage after mirror (two disks in > spare), 2.5TB L2ARC and 64GB Zil, all fit into a single 5U box. > > > > Both L2ARC and Zil shares the same disks (striped) due to bandwidth > requirements. Each SSD has a theoretical performance of 40-50k IOPS on > 4k read/write scenario with 70/30 distribution. Now, I know that you > should have mirrored Zil for safety, but the entire box are > synchronized with an active standby on a different site location (18km > distance - round trip of 0.16ms + equipment latency). So in case the > Zil in Site A takes a fall, or the motherboard/disk group/motherboard > dies - we still have safety. > > I expect that you need more space for L2ARC and a lot less for Zil. > Furthmore, you'd be better served by an even lower latency/higher IOPs > ZIL. If you're going to spend this kind of cash, I think I'd recommend > at least one or two DDR Drive X1 units or something similar. While not > very big, you don't need much to get a huge benefit from the ZIL, and I > think the vastly superior IOPS of these units will pay off in the end. >
What about the ZIL bandwidth in this case? I mean, could I stripe across multiple devices to be able to handle higher throughput? Otherwise I would still be limited to the performance of the unit itself (155 MB/s). > > > > DDT requirements for dedupe on 16k blocks should be about 640GB when > main pool are full (capacity). > > Dedup is not always a win, I think. I'd look hard at your data and > usage to determine whether to use it. > > -- Garrett -Arve _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
