Re: [ceph-users] Help with SSDs
Hi Mark, On 18.12.2014 07:15, Mark Kirkwood wrote: While you can't do much about the endurance lifetime being a bit low, you could possibly improve performance using a journal *file* that is located on the 840's (you'll need to symlink it - disclaimer - have not tried this myself, but will experiment if you are interested). Slightly different open() options are used in this case and these cheaper consumer SSD seem to work better with them. I had the symlink-file method before, (with different SSDs) but the performance was much better after changing to partitions. I try fist some different consumer SSDs with journal as file and end now with DC S3700 with partitions. Udo ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
The effect of this is *highly* dependent to the SSD make/model. My m550 work vastly better if the journal is a file on a filesystem as opposed to a partition. Obviously the Intel S3700/S3500 are a better choice - but the OP has already purchased Sammy 840's, so I'm trying to suggest options to try that don't require him to buy new SSDs! Cheers Mark On 18/12/14 21:28, Udo Lembke wrote: On 18.12.2014 07:15, Mark Kirkwood wrote: While you can't do much about the endurance lifetime being a bit low, you could possibly improve performance using a journal *file* that is located on the 840's (you'll need to symlink it - disclaimer - have not tried this myself, but will experiment if you are interested). Slightly different open() options are used in this case and these cheaper consumer SSD seem to work better with them. I had the symlink-file method before, (with different SSDs) but the performance was much better after changing to partitions. I try fist some different consumer SSDs with journal as file and end now with DC S3700 with partitions. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
On Thu, 18 Dec 2014 10:05:20 PM Mark Kirkwood wrote: My m550 work vastly better if the journal is a file on a filesystem as opposed to a partition. Any particular filesystem? ext4? xfs? or doesn't matter? -- Lindsay signature.asc Description: This is a digitally signed message part. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
On Thu, 18 Dec 2014 10:05:20 PM Mark Kirkwood wrote: The effect of this is *highly* dependent to the SSD make/model. My m550 work vastly better if the journal is a file on a filesystem as opposed to a partition. Obviously the Intel S3700/S3500 are a better choice - but the OP has already purchased Sammy 840's, so I'm trying to suggest options to try that don't require him to buy new SSDs! I have 120GB Samsung 840 EVO's with 10GB journal partitions and just gave this a go. No real change unfortunately :( using rados bench. However it does make experimenting with different journal sizes easier. -- Lindsay signature.asc Description: This is a digitally signed message part. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
On 19/12/14 03:01, Lindsay Mathieson wrote: On Thu, 18 Dec 2014 10:05:20 PM Mark Kirkwood wrote: The effect of this is *highly* dependent to the SSD make/model. My m550 work vastly better if the journal is a file on a filesystem as opposed to a partition. Obviously the Intel S3700/S3500 are a better choice - but the OP has already purchased Sammy 840's, so I'm trying to suggest options to try that don't require him to buy new SSDs! I have 120GB Samsung 840 EVO's with 10GB journal partitions and just gave this a go. No real change unfortunately :( using rados bench. However it does make experimenting with different journal sizes easier. Pity. If you used xfs you can try tweaking some of the mkfs options...but I doubt they will make too much difference. Looking at the data specs for the 840, it does not seem to have any on board capacitors. If it did you could risk switching off xfs write barriers...which would probably make a big difference. You could try switching *off* the write cache (just in case the 840 behaves like my m550's and gets - oddly- 2x faster for sync writes in that case)! However disabling the write cache may *considerably* decrease disk lifetime, so if the setting helps in yur case, you'll need to conduct some experiments to measure by how much the lifetime is gonna be impacted. Cheers Mark ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Help with SSDs
Hey there, Is there a good work around if our SSDs are not handling D_SYNC very well? We invested a ton of money into Samsung 840 EVOS and they are not playing well with D_SYNC. Would really appreciate the help! Thank you, Bryson ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
Hello, On Tue, 16 Dec 2014 08:58:23 -0700 Bryson McCutcheon wrote: Hey there, Is there a good work around if our SSDs are not handling D_SYNC very well? We invested a ton of money into Samsung 840 EVOS and they are not playing well with D_SYNC. Would really appreciate the help! Baring any reply from the developers, I'm pretty sure the answer is no, as D_SYNC is a vital part of the journaling process. You _may_ get away with a hardware cache RAID controller (some models/makers will use that cache in JBOD mode, too). And while I very much can sympathize with you (I have dozens of new Toshiba DTxx drives sitting on a pile that perform quite nicely for a week before they go into half to 10% speed mode for at least 10 hours) and don't want to sound snarky, penultimately you get what you pay for. Journals on consumer SSDs are a bad idea. Aside form the performance issues you're experiencing, Samsung doesn't really give a TBW value for them, the estimation based on their claims is about 100TBW for the 120GB model. While not bad, it's something I'd be weary about with regards to long term usage. http://ssdendurancetest.com/ssd-endurance-test-report/Samsung-840-EVO-120 The only reliable game in town at this point in time are Intel DC S3700 models, the 200GB model for example has a TBW of 1.8PB and will keep its speed w/o the need for TRIM or massive underprovisioning. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
On 12/17/2014 02:58 AM, Bryson McCutcheon wrote: Is there a good work around if our SSDs are not handling D_SYNC very well? We invested a ton of money into Samsung 840 EVOS and they are not playing well with D_SYNC. Would really appreciate the help! Just in case it's linked with the recent performance issue: are they upgraded/refreshed with the latest firmware? (http://www.samsung.com/au/consumer/pc-peripherals/solid-state-drive/ssd-840-evo/MZ-7TE1T0BW) I have EVOs too, what to you mean by not playing well with D_SYNC? Is there something I can test on my side to compare results with you, as I have mine flashed? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
Hi Mikaël, I have EVOs too, what to you mean by not playing well with D_SYNC? Is there something I can test on my side to compare results with you, as I have mine flashed? http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ described how test the ssd-performance for an journal ssd (your ssd will be overwritten!!). Udo ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
what to you mean by not playing well with D_SYNC? Hi, check this blog: http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ - Mail original - De: Mikaël Cluseau mclus...@isi.nc À: Bryson McCutcheon brysonmccutch...@gmail.com, ceph-users ceph-users@lists.ceph.com Envoyé: Jeudi 18 Décembre 2014 03:16:07 Objet: Re: [ceph-users] Help with SSDs On 12/17/2014 02:58 AM, Bryson McCutcheon wrote: Is there a good work around if our SSDs are not handling D_SYNC very well? We invested a ton of money into Samsung 840 EVOS and they are not playing well with D_SYNC. Would really appreciate the help! Just in case it's linked with the recent performance issue: are they upgraded/refreshed with the latest firmware? (http://www.samsung.com/au/consumer/pc-peripherals/solid-state-drive/ssd-840-evo/MZ-7TE1T0BW) I have EVOs too, what to you mean by not playing well with D_SYNC? Is there something I can test on my side to compare results with you, as I have mine flashed? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Help with SSDs
Looking at the blog, I notice he disabled the write cache before the tests: doing this on my m550 resulted in *improved* dsync results (300 IOPS - 700 IOPS) still not great obviously, but ... interesting. So do experiment with the settings to see if you can get the 840's working better for you. While you can't do much about the endurance lifetime being a bit low, you could possibly improve performance using a journal *file* that is located on the 840's (you'll need to symlink it - disclaimer - have not tried this myself, but will experiment if you are interested). Slightly different open() options are used in this case and these cheaper consumer SSD seem to work better with them. Regards Mark On 18/12/14 18:08, Alexandre DERUMIER wrote: what to you mean by not playing well with D_SYNC? Hi, check this blog: http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
