Hello, As others said, it depends on your use case and expected write load. If you search the ML archives, you will find that there can be SEVERE write amplification with Ceph, something to very much keep in mind.
You should run tests yourself before deploying things and committing to a hardware that won't cut the mustard. I did this comparison for a project (involving DRBD, not Ceph) 3 months ago: --- Model Size Endurance Cost Samsung 845DC EVO 960GB 600TBW 700USD TBW/$=1.16 Intel DC S3700 800GB 7300TBW 1500USD TBW/$=0.20 --- In that particular case the Samsung made the grade, as the expected writes per year and SSD are less than 60TB. Make those calculation for the specific SSDs you have in mind. Cheap initial costs may come back to bite you in the behind later. With Ceph, I'd be _very_ uncomfortable putting data on consumer SSDs. Aside from the ease of mind that the Intel DC S3700 (or maybe the Samsung 845DC Pro, not tested myself yet) give you, there's the overall better performance (and consistently so, no going to sleep for garbage collection). On top of the SMART monitoring you'll probably also be forced use fstrim on these SSDs to keep their performance (such as it is) from degrading. Christian On Wed, 28 Jan 2015 00:30:04 +0000 Quenten Grasso wrote: > Hi Nick, > > Agreed, I see your point of basically once your past the 150TBW or > whatever that number maybe, your just waiting for failure effectively > but aren't we anyway? > > I guess it depends on your use case at the end of the day. I wonder what > the likes of Amazon, Rackspace etc are doing in the way of SSD's, either > they are buying them so cheap per GB due to the "volume" or they are > possibly using "consumer grade" SSD'. > > hmm.. using consumer grade SSD's it may be an interesting option if you > have descent monitoring and alerting using SMART you should be able to > still see how much spare flash you have available. As suggested by Wido > using multiple brands would help remove the possible cascading failure > affect which I guess we all should be doing anyway on our spinners. > > I guess we have to decide is it worth the extra effort in the long run > vs running enterprise ssds. > > Regards, > Quenten Grasso > > From: Nick Fisk [mailto:n...@fisk.me.uk] > Sent: Saturday, 24 January 2015 7:33 PM > To: Quenten Grasso; ceph-users@lists.ceph.com > Subject: RE: Consumer Grade SSD Clusters > > Hi Quenten, > > There is no real answer to your question. It really depends on how busy > your storage will be and particularly if it is mainly reads or writes. > > I wouldn't pay too much attention to that SSD endurance test, whilst > it's great to know that they have a lot more headroom than their > official spec's, you run the risk of having a spectacular multiple disk > failure if you intend to run them all that high. You can probably > guarantee that as 1 SSD starts to fail the increase in workload to > re-balance the cluster will cause failures on the rest. > > I guess it really comes down to how important is the availability of > your data. Whilst an average pc user might balk at the price of paying 4 > times per GB more for a S3700 SSD, in the enterprise world they are > still comparatively cheap. > > The other thing you need to be aware of is that most consumer SSD's > don't have power loss protection, again if you are mainly doing reads > and cost is more important than availability, there may be an argument > to use them. > > Nick > > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > Quenten Grasso Sent: 24 January 2015 09:13 > To: ceph-users@lists.ceph.com > Subject: [ceph-users] Consumer Grade SSD Clusters > > Hi Everyone, > > Just wondering if anyone has had any experience in using consumer grade > SSD's for a Ceph cluster? > > I came across this article > http://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte/3<http://xo4t.mjt.lu/link/xo4t/gg573yr/1/QRjiN_2beI5qST5ggOanaQ/aHR0cDovL3RlY2hyZXBvcnQuY29tL3Jldmlldy8yNjUyMy90aGUtc3NkLWVuZHVyYW5jZS1leHBlcmltZW50LWNhc3VhbHRpZXMtb24tdGhlLXdheS10by1hLXBldGFieXRlLzM> > > They have been testing different SSD's write endurance and they have > been able to write up to 1PB+ to a Samsung 840 Pro 256GB which is only > "rated" at 150TBW and of course other SSD's have failed well before > 1PBW, So defiantly worth a read. > > So I've been thinking about using consumer grade SSD's for OSD's and > Enterprise SSD's for journals. > > Reasoning is enterprise SSD's are a lot faster at journaling then > consumer grade drives plus this would effectively half the overall write > requirements on the consumer grade disks. > > This also could be a cost effective alternative to using enterprise > SSD's as OSD's however it seems if your happy to use 2 x replication > it's a pretty good cost saving however 3x replication not so much. > > Cheers, > Quenten Grasso > > > -- Christian Balzer Network/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
