[ceph-users] Jewel/Luminous Filestore/Bluestore for a new cluster
Hi again, I've been happily using both Hammer and Jewel with SSD journals and spinning disk Filestore OSDs for several years now and, as per my other email, I'm about to purchase hardware to build a new (separate) production cluster. I intend to use the same mixture of SSD for journals (or DB/WAL) and spinning disks for Filestore/Bluestore data as per my existing cluster. * What's the recommendation for what to deploy? I have a feeling the answer is going to be Luminous (as that's current LTS) and Bluestore (since that's the default in Luminous) but several recent threads and comments on this list make me doubt whether that would be a good choice right now. * Is using Bluestore and integrated DB/WAL (without SSDs at all) a reasonable option for those used to the performance of SSD Journals + spinning disk Filestore OSDs? Thanks very much in advance for any advice. Simon. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Jewel/Luminous Filestore/Bluestore for a new cluster
On 05/30/2018 09:20 PM, Simon Ironside wrote: > * What's the recommendation for what to deploy? > > I have a feeling the answer is going to be Luminous (as that's current > LTS) and Bluestore (since that's the default in Luminous) but several > recent threads and comments on this list make me doubt whether that > would be a good choice right now. Indeed Why would you deploy a Jewel cluster, which is almost 3 majors versions away ? Bluestore is also the good answer It works well, have many advantages, and is simply the future of Ceph ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Jewel/Luminous Filestore/Bluestore for a new cluster
On 30/05/18 20:35, Jack wrote: Why would you deploy a Jewel cluster, which is almost 3 majors versions away ? Bluestore is also the good answer It works well, have many advantages, and is simply the future of Ceph Indeed, and normally I wouldn't even ask, but as I say there's been some comments/threads recently that make me doubt the obvious Luminous + Bluestore path. A few that stand out in my memory are: * "Useless due to http://tracker.ceph.com/issues/22102"; [1] * OSD crash with segfault Luminous 12.2.4 [2] [3] [4] There are others but those two stuck out for me. I realise that people will generally only report problems rather than "I installed ceph and everything went fine!" stories to this list but it was enough to motivate me to ask if Luminous/Bluestore was considered a good choice for a fresh install or if I should wait a bit. Thanks, Simon. [1] http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026339.html [2] http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-March/025373.html [3] http://tracker.ceph.com/issues/23431 [4] http://tracker.ceph.com/issues/23352 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Jewel/Luminous Filestore/Bluestore for a new cluster
We have been running a Luminous.04 + Bluestor for about 3 months in production. All the daemons run as docker containers and were installed using ceph-ansible. 540 spinning drives with journal/wal/db on the same drive spread across 9 hosts. Using librados object interface directly with steady 100MB/s writes to it. We have not observed any major issues. We have had occasional OSD daemon crashes due to an assert which is a known bug but the cluster recovered without any intervention each time. All the nodes have been rebooted 2-3 times due to CoreOS updates and no issues with that either. If you have any specific questions related to the cluster, please post them on this thread. Subhachandra On Wed, May 30, 2018 at 1:06 PM, Simon Ironside wrote: > On 30/05/18 20:35, Jack wrote: > >> Why would you deploy a Jewel cluster, which is almost 3 majors versions >> away ? >> Bluestore is also the good answer >> It works well, have many advantages, and is simply the future of Ceph >> > > Indeed, and normally I wouldn't even ask, but as I say there's been some > comments/threads recently that make me doubt the obvious Luminous + > Bluestore path. A few that stand out in my memory are: > > * "Useless due to http://tracker.ceph.com/issues/22102"; [1] > * OSD crash with segfault Luminous 12.2.4 [2] [3] [4] > > There are others but those two stuck out for me. I realise that people > will generally only report problems rather than "I installed ceph and > everything went fine!" stories to this list but it was enough to motivate > me to ask if Luminous/Bluestore was considered a good choice for a fresh > install or if I should wait a bit. > > Thanks, > Simon. > > [1] http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018- > May/026339.html > [2] http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018- > March/025373.html > [3] http://tracker.ceph.com/issues/23431 > [4] http://tracker.ceph.com/issues/23352 > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Jewel/Luminous Filestore/Bluestore for a new cluster
On 05/06/18 01:14, Subhachandra Chandra wrote: We have not observed any major issues. We have had occasional OSD daemon crashes due to an assert which is a known bug but the cluster recovered without any intervention each time. All the nodes have been rebooted 2-3 times due to CoreOS updates and no issues with that either. Hi Subhachandra, Thanks for your answer, it's this sort of stuff that worries me. My Filestore OSD daemons on Hammer & Jewel don't crash at all so this sounds like a regression and I should wait before deploying Luminous/Bluestore. Simon ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
