As nobody chimed in, let me reply inline.
Best Regards,Strahil Nikolov
On Sunday, April 23, 2023, 2:35 AM, Peter P <peter.a.pfeif...@gmail.com> wrote:
Good afternoon,
I am looking for additional information about glusterfs, arbiters and
thin-arbiters. The current stable release is gluster 11, so I am considering
that version for deployment. My planned setup is: 4 storage servers in
distributed + replicated mode.
Server1, server2 : replica 2, arbiter 1
Server3, server4 : replica 2, arbiter 1
Since having replica 2 is not recommended due to split-brains I will have an
arbiter.
Generic questions:
- Is arbiter or thin-arbiter recommended in a production, large volume
storage environment?
- Both were introduced a long time ago. Most users prefer full arbiter, so
healing is far more optimal (only changed files will be healed).
- Is thin arbiter code stable and deployment ready?
- I know that it’s in use but full arbiter has been introduced earlier and has
a wider adoption.
- Arbiter is file based and stores metadata for each file. In this scenario
I would at least double the default inode count on the arbiter server.
Thin-arbiter on the other hand is brick based but I have not found enough
information if its inode usage is as inode hungry as the arbiter configuration.
I am thinking, it should not be as it is brick based. So do I need to increase
the inode count when using thin-arbiters? If yes, what is recommended?
- Full arbiter is sensitive to network latency and disk speed (a lot of small
I/O for those inodes). Increase macpct (XFS) on arbiter bricks and prefer using
a SSD/NVME. As full arbiter doesn’t store any data , you can set the maxpct to
around 75%- Thin arbiter doesn’t have a brick and when you create it, you just
specify the replica id file ( see
https://docs.gluster.org/en/main/Administrator-Guide/Thin-Arbiter-Volumes/ )
- I've read old bug reports reporting that thin arbiter was not ready to
serve multiple trusted pools. Is this still the case? I may configure multiple
trusted pools in the future.
- I saw Kadalu uses their own thin arbiter and I never saw issues. I doubt I
was the only one using it, so it should be fine.
- I have many linux boxes running different linux distributions and
releases. Ideally the assortment of boxes would mount the same gluster
pool/volume. I looked for information about older versions of gluster clients
running on a range of older distributions mounting the most recent gluster 11
pool/volume? Does that work? Can gluster client (version 10, 9, 8, 7, etc.)
mount gluster 11 volume and run without significant issues? I understand that
older versions of client will not have the most recent features. Most recent
features aside, is such configuration supported/stable?
- For that purpose gluster has 2 settings:cluster.max-op-version -> the max
compatibility version you can set your cluster based of the oldest client’s
versioncluster.op-version -> the cluster’s compatibility versionAs long you
keep the cluster.op-version compatible with your client - it should work.
Thin-arbiter approach: If I go with the thin-arbiter configuration I will use
a 5th server as this server can be outside of the trusted pool and can be
shared among multiple trusted pools
Server1, server2: replica 2, thin-arbiter server5
Server3, server4: replica 2, thin-arbiter server5
Old arbiter approach: If I go with the older arbiter configuration, I am
considering using 2 of the storage servers to act as both replica and an
arbiter. Is that configuration possible/supported and reasonable?
Server1, server2: replica 2, arbiter server3
Server3, server4: replica 2, arbiter server1
- Yes, as long as you have a dedicated brick (in this example server3 should
have a data brick and arbiter brick)
In this configuration, I am considering using server3 to be arbiter for
server{1,2} replica 2, and using server1 to be arbiter for server{3,4} replica
2.
Questions:
- Is this a reasonable/recommended configuration?
- It’s used quite often
- Should the arbiter metadata folder be inside or outside of the volume?
- In detail. Say server{1,2} replica has 1 brick each /gluster/brick1 with
/gfs1vol1 as the volume
- Should the arbiter metadata folder location be:
/gluster/arbiter/gfs1vol1 (outside of the volume path) or
/gfs1vol1/arbiter1/ (inside the volume path)
- Always keep bricks as separate mount points. For example:/dev/vg/lv mounted
on /bricks/databricks with directory vol1/brick1/dev/vg/lv2 mounted on
/bricks/arbiterbricks with directory vol1/arbiterbrick1
The idea is that if the device is not mounted, the brick directory will be
missing and the mess will be far less.
Thank you for your thoughts,
Peter
| | Virus-free.www.avg.com |
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
