Hello Robert & Christian, First, thank you for the general considerations, 3 and 3.extra has been ruled out.
> A simple way to make 1) and 2) cheaper is to use AMD CPUs, they will do > just fine at half the price with these loads. > If you're that tight on budget, 64GB RAM will do fine, too. > > I assume you're committed to 10GbE in your environment, at least when it > comes to the public side. > I have found Infiniband cheaper (especially when it comes to switches) > and faster that 10GbE. > We decided to go with 10GbE on the storage side to consolidate the 10GbE external network connectivity requirement with the storage networking, and not use two separate technologies/switches/NICs in the compute and storage nodes. > Looking purely at bandwidth (sequential writes), your proposals are all > underpowered when it comes to the ratio of SSD journals to HDDs and the > available network bandwidth. > For example with 1) you have up to 2GB/s of inbound writes from the > network and about 1.7GB/s worth on your HDDs, but just 700GB/s on your > SSDs. > Even if you're more interested in IOPS (as you probably should), it > feels like a waste. > 2) with 4 SSDs (or bigger ones that are faster) would make a decent > storage node it my book. This is a very good point that I totally overlooked. I concentrated more on the IOPS alignment plus write durability, and forgot to check the sequential write bandwidth. The 400GB Intel S3700 is a lot more faster but double the price (around $950) compared to the 200GB. Maybe I would be better off using enterprise SLC SSDs for journals? For example OCZ Deneva 2 C 60GB SLC costs around $640, and have 75K write IOPS and ~510MB/s write bandwidth by spec. Cheers, Benjamin _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
