Gagan: Throwing my $0.02 in --
It depends on the system environment of how you are planning on deploying Gluster (and/or Ceph). I have Ceph running on my three node HA Proxmox cluster using three OASLOA Mini PCs that only has the Intel N95 Processor (4-core/4-thread) with 16 GB of RAM and a cheap Microcenter store brand 512 GB NVMe M.2 2230 SSD and my Ceph cluster has been running without any issues. As someone else mentioned, to state or claim that Ceph is "hardware demanding" isn't wholly accurate. As for management, you can install the ceph-mgr-dashboard package (there is a video that apalrd's adventures put together on YouTube which goes over the installation process for this package, if you're running Debian and/or Proxmox (which runs on top of Debian anyways).) From there, you can use said Ceph manager dashboard to do everything else, so that you don't have to deploy Ceph via the CLI. I was able to create my erasure coded CRUSH rules using the dashboard, and then create my RBD pool and also my CephFS pool as well. (The metadata needs a replicate CRUSH rule, but the data itself can use erasure coded CRUSH rule.) If your environment is such that you can do this, then Ceph might be a better option for you. If you look at the benchmarks that tech YouTuber ElectronicsWizardry ran, ZFS is actually not all that performant. But what ZFS is good for are some of the other features like snapshots, replications, and it's copy-on-write schema for modifying files (which again, based on the testing that ElectronicsWizardry ran, does indeed create a write amplification effect as a result of the copy-on-write architecture) Conversely, if you're looking for reliability, the more nodes that you have in the Ceph cluster, the more reliable and resilient to failures the Ceph backend will be. Thanks. Sincerely, Ewen ________________________________ From: Gluster-users <[email protected]> on behalf of gagan tiwari <[email protected]> Sent: April 17, 2025 2:14 AM To: Alexander Schreiber <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: [Gluster-users] Gluster with ZFS HI Alexander, Thanks for the update. Initially, I also thought of deploying Ceph but ceph is quite difficult to set-up and manage. Moreover, it's also hardware demanding. I think it's most suitable for a very large set-up with hundreds of clients. What do you think of MooseFS ? Have you or anyone else tried MooseFS. If yes, how was its performance? Thanks, Gagan On Thu, Apr 17, 2025 at 1:45 PM Alexander Schreiber <[email protected]<mailto:[email protected]>> wrote: On Thu, Apr 17, 2025 at 09:40:08AM +0530, gagan tiwari wrote: > Hi Guys, > We have been using OpenZFS in our HPC environment for > quite some time. And OpenZFS was going fine. > > But we are now running into scalability issues since OpenZFS can't be > scaled out. Since ZFS is a local FS, you are essentially limited to how much storage you can stick into one machine, yes. > So, I am planning to use Gluster on top of OpenZFS. I don't think that is giving you the kind of long term scalability you might expect. > So, I wanted to know if anyone has tried it. if yes, how it was and any > deployment guide for it. I'm running GlusterFS in a small cluster for backup storage. > We have an HPC environment . Data security and extremely fast read > performance is very important for us. > > So, please advise. For that use case I would actually recommend Ceph over GlusterFS, since that can be pretty easily scaled out to very large setups, e.g. CERN is using multiple Ceph clusters sized at several PB and their use cases usually include very fast I/O. Another concern is that Ceph is being quite actively developed whereas GlusterFS development seems to have slowed down to ... not much, these days. Kind regards, Alex. -- "Opportunity is missed by most people because it is dressed in overalls and looks like work." -- Thomas A. Edison
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
