Hi, We are currently building a storage box based on OpenSolaris/Nexenta using ZFS. Our hardware specifications are as follows:
Quad AMD G34 12-core 2.3 GHz (~110 GHz) 10 Crucial RealSSD (6Gb/s) 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS expanders LSI2008SAS (two 4x ports) Mellanox InfiniBand 40 Gbit NICs 128 GB RAM This setup gives us about 40TB storage after mirror (two disks in spare), 2.5TB L2ARC and 64GB Zil, all fit into a single 5U box. Both L2ARC and Zil shares the same disks (striped) due to bandwidth requirements. Each SSD has a theoretical performance of 40-50k IOPS on 4k read/write scenario with 70/30 distribution. Now, I know that you should have mirrored Zil for safety, but the entire box are synchronized with an active standby on a different site location (18km distance - round trip of 0.16ms + equipment latency). So in case the Zil in Site A takes a fall, or the motherboard/disk group/motherboard dies - we still have safety. DDT requirements for dedupe on 16k blocks should be about 640GB when main pool are full (capacity). Without going into details about chipsets and such, do any of you on this list have any experience with a similar setup and can share with us your thoughts, do's and dont's, and any other information that could be of help while building and configuring this? What I want to achieve is 2 GB/s+ NFS traffic against our ESX clusters (also InfiniBand-based), with both dedupe and compression enabled in ZFS. Let's talk moon landings. Regards, Arve -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
