> It is very hard to compare them because they are structurally very different. > For example, GlusterFS performance will depend *a lot* on the underlying file > system performance. Ceph eliminated that factor by using Bluestore. > Ceph is very well performing for VM storage, since it's block based and as > such optimized for that. I haven't tested CephFS a lot (I used it but only > for very small storage) so I cannot speak for its performance, but I am > guessing it's not ideal. For large amount of files thus GlusterFS is still a > good choice.
Was your experience above based on using a sharded volume or a normal one? When we worked with virtual machine images, we followed the volume sharding advice. I don't have a comparison for Ceph handy. I was just curious. It worked so well for us (but maybe our storage is "too good") that we found it hard to imagine it could be improved much. This was a simple case though of a single VM, 3 gluster servers, a sharded volume, and a raw virtual machine image. Probably a simpler case than yours. Thank you for writing this and take care, Erik > > One *MAJOR* advantage of Ceph over GlusterFS is tooling. Ceph's > self-analytics, status reporting and problem fixing toolset is just so far > beyond GlusterFS that it's really hard for me to recommend GlusterFS for any > but the most experienced sysadmins. It does come with the type of > implementation Ceph has chosen that they have to have such good tooling > (because honestly, poking around in binary data structures really wouldn't be > practical for most users), but whenever I had a problem with Ceph the > solution was just a couple of command line commands (even if it meant to > remove a storage device, wipe it and add it back), where with GlusterFS it > means poking around in the .glusterfs directory, looking up inode numbers, > extended attributes etc. which is a real pain if you have a > multi-million-file filesystem to work on. And that's not even with sharding > or distributed volumes. > > Also, Ceph has been a lot more stable that GlusterFS for us. The amount of > hand-holding GlusterFS needs is crazy. With Ceph, there is this one bug (I > think in certain Linux kernel versions) where it sometimes reads only zeroes > from disk and complains about that and then you have to restart that OSD to > not run into problems, but that's one "swatch" process on each machine that > will do that automatically for us. I have run some Ceph clusters for several > years now and only once or twice I had to deal with problems. The several > GlusterFS clusters we operate constantly run into troubles. We now shut down > all GlusterFS clients before we reboot any GlusterFS node because it was near > impossible to reboot a single node without running into unrecoverable > troubles (heal entries that will not heal etc.). With Ceph we can achieve > 100% uptime, we regularly reboot our hosts one by one and some minutes later > the Ceph cluster is clean again. > > If others have more insights I'd be very happy to hear them. > > Stefan > > > ----- Original Message ----- > > Date: Tue, 16 Jun 2020 20:30:34 -0700 > > From: Artem Russakovskii <archon...@gmail.com> > > To: Strahil Nikolov <hunter86...@yahoo.com> > > Cc: gluster-users <gluster-users@gluster.org> > > Subject: Re: [Gluster-users] State of Gluster project > > Message-ID: > > <CAD+dzQdf_TiPBSDj57hY=t8AQ=macrxinpx7iu4hmuxnmo+...@mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Has anyone tried to pit Ceph against gluster? I'm curious what the ups and > > downs are. > > > > On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov <hunter86...@yahoo.com> wrote: > > > >> Hey Mahdi, > >> > >> For me it looks like Red Hat are focusing more on CEPH than on Gluster. > >> I hope the project remains active, cause it's very difficult to find a > >> Software-defined Storage as easy and as scalable as Gluster. > >> > >> Best Regards, > >> Strahil Nikolov > >> > >> ?? 17 ??? 2020 ?. 0:06:33 GMT+03:00, Mahdi Adnan <ma...@sysmin.io> ??????: > >> >Hello, > >> > > >> > I'm wondering what's the current and future plan for Gluster project > >> >overall, I see that the project is not as busy as it was before "at > >> >least > >> >this is what I'm seeing" Like there are fewer blogs about what the > >> >roadmap > >> >or future plans of the project, the deprecation of Glusterd2, even Red > >> >Hat > >> >Openshift storage switched to Ceph. > >> >As the community of this project, do you feel the same? Is the > >> >deprecation > >> >of Glusterd2 concerning? Do you feel that the project is slowing down > >> >somehow? Do you think Red Hat is abandoning the project or giving fewer > >> >resources to Gluster? > >> ________ > >> > >> > >> > >> Community Meeting Calendar: > >> > >> Schedule - > >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > >> Bridge: https://bluejeans.com/441850968 > >> > >> Gluster-users mailing list > >> Gluster-users@gluster.org > >> https://lists.gluster.org/mailman/listinfo/gluster-users > >> > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > <http://lists.gluster.org/pipermail/gluster-users/attachments/20200616/a1d0f142/attachment-0001.html > > > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
