As nobody chimed in, let me reply inline. Best Regards,Strahil Nikolov
On Sunday, April 23, 2023, 2:35 AM, Peter P <peter.a.pfeif...@gmail.com> wrote: Good afternoon, I am looking for additional information about glusterfs, arbiters and thin-arbiters. The current stable release is gluster 11, so I am considering that version for deployment. My planned setup is: 4 storage servers in distributed + replicated mode. Server1, server2 : replica 2, arbiter 1 Server3, server4 : replica 2, arbiter 1 Since having replica 2 is not recommended due to split-brains I will have an arbiter. Generic questions: - Is arbiter or thin-arbiter recommended in a production, large volume storage environment? - Both were introduced a long time ago. Most users prefer full arbiter, so healing is far more optimal (only changed files will be healed). - Is thin arbiter code stable and deployment ready? - I know that it’s in use but full arbiter has been introduced earlier and has a wider adoption. - Arbiter is file based and stores metadata for each file. In this scenario I would at least double the default inode count on the arbiter server. Thin-arbiter on the other hand is brick based but I have not found enough information if its inode usage is as inode hungry as the arbiter configuration. I am thinking, it should not be as it is brick based. So do I need to increase the inode count when using thin-arbiters? If yes, what is recommended? - Full arbiter is sensitive to network latency and disk speed (a lot of small I/O for those inodes). Increase macpct (XFS) on arbiter bricks and prefer using a SSD/NVME. As full arbiter doesn’t store any data , you can set the maxpct to around 75%- Thin arbiter doesn’t have a brick and when you create it, you just specify the replica id file ( see https://docs.gluster.org/en/main/Administrator-Guide/Thin-Arbiter-Volumes/ ) - I've read old bug reports reporting that thin arbiter was not ready to serve multiple trusted pools. Is this still the case? I may configure multiple trusted pools in the future. - I saw Kadalu uses their own thin arbiter and I never saw issues. I doubt I was the only one using it, so it should be fine. - I have many linux boxes running different linux distributions and releases. Ideally the assortment of boxes would mount the same gluster pool/volume. I looked for information about older versions of gluster clients running on a range of older distributions mounting the most recent gluster 11 pool/volume? Does that work? Can gluster client (version 10, 9, 8, 7, etc.) mount gluster 11 volume and run without significant issues? I understand that older versions of client will not have the most recent features. Most recent features aside, is such configuration supported/stable? - For that purpose gluster has 2 settings:cluster.max-op-version -> the max compatibility version you can set your cluster based of the oldest client’s versioncluster.op-version -> the cluster’s compatibility versionAs long you keep the cluster.op-version compatible with your client - it should work. Thin-arbiter approach: If I go with the thin-arbiter configuration I will use a 5th server as this server can be outside of the trusted pool and can be shared among multiple trusted pools Server1, server2: replica 2, thin-arbiter server5 Server3, server4: replica 2, thin-arbiter server5 Old arbiter approach: If I go with the older arbiter configuration, I am considering using 2 of the storage servers to act as both replica and an arbiter. Is that configuration possible/supported and reasonable? Server1, server2: replica 2, arbiter server3 Server3, server4: replica 2, arbiter server1 - Yes, as long as you have a dedicated brick (in this example server3 should have a data brick and arbiter brick) In this configuration, I am considering using server3 to be arbiter for server{1,2} replica 2, and using server1 to be arbiter for server{3,4} replica 2. Questions: - Is this a reasonable/recommended configuration? - It’s used quite often - Should the arbiter metadata folder be inside or outside of the volume? - In detail. Say server{1,2} replica has 1 brick each /gluster/brick1 with /gfs1vol1 as the volume - Should the arbiter metadata folder location be: /gluster/arbiter/gfs1vol1 (outside of the volume path) or /gfs1vol1/arbiter1/ (inside the volume path) - Always keep bricks as separate mount points. For example:/dev/vg/lv mounted on /bricks/databricks with directory vol1/brick1/dev/vg/lv2 mounted on /bricks/arbiterbricks with directory vol1/arbiterbrick1 The idea is that if the device is not mounted, the brick directory will be missing and the mess will be far less. Thank you for your thoughts, Peter | | Virus-free.www.avg.com | ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users