Hi Stefan >>Have you already done tests how he performance changes with bluestore >>while putting all 3 block devices on the same ssd?
I'm going to test bluestore with 3 nodes , 18 x intel s3610 1,6TB in coming weeks. I'll send results on the mailing. ----- Mail original ----- De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> À: "Christian Balzer" <ch...@gol.com>, "ceph-users" <ceph-users@lists.ceph.com> Envoyé: Jeudi 7 Septembre 2017 08:03:31 Objet: Re: [ceph-users] PCIe journal benefit for SSD OSDs Hello, Am 07.09.2017 um 03:53 schrieb Christian Balzer: > > Hello, > > On Wed, 6 Sep 2017 09:09:54 -0400 Alex Gorbachev wrote: > >> We are planning a Jewel filestore based cluster for a performance >> sensitive healthcare client, and the conservative OSD choice is >> Samsung SM863A. >> > > While I totally see where you're coming from and me having stated that > I'll give Luminous and Bluestore some time to mature, I'd also be looking > into that if I were being in the planning phase now, with like 3 months > before deployment. > The inherent performance increase with Bluestore (and having something > that hopefully won't need touching/upgrading for a while) shouldn't be > ignored. Yes and that's the point where i'm currently as well. Thinking about how to design a new cluster based on bluestore. > The SSDs are fine, I've been starting to use those recently (though not > with Ceph yet) as Intel DC S36xx or 37xx are impossible to get. > They're a bit slower in the write IOPS department, but good enough for me. I've never used the Intel DC ones but always the Samsung are the Intel really faster? Have you disabled te FLUSH command for the Samsung ones? They don't skip the command automatically like the Intel do. Sadly the Samsung SM863 got more expensive over the last months. They were a lot cheaper in the first month of 2016. May be the 2,5" optane intel ssds will change the game. >> but was wondering if anyone has seen a positive >> impact from also using PCIe journals (e.g. Intel P3700 or even the >> older 910 series) in front of such SSDs? >> > NVMe journals (or WAL and DB space for Bluestore) are nice and can > certainly help, especially if Ceph is tuned accordingly. > Avoid non DC NVMes, I doubt you can still get 910s, they are officially > EOL. > You want to match capabilities and endurances, a DC P3700 800GB would be > an OK match for 3-4 SM863a 960GB for example. That's a good point but makes the cluster more expensive. Currently while using filestore i use one SSD for journal and data which works fine. With bluestore we've block, db and wal so we need 3 block devices per OSD. If we need one PCIe or NVMe device per 3-4 devices it get's much more expensive per host - currently running 10 OSDs / SSDs per Node. Have you already done tests how he performance changes with bluestore while putting all 3 block devices on the same ssd? Greets, Stefan _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
