Re: [Gluster-users] To RAID or not to RAID...

Hi, our old setup is not really comparable, but i thought i'd drop some lines... we once had a Distributed-Replicate setup with 4 x 3 = 12 disks (10 TB hdd). Simple JBOD, every disk == brick. Was running pretty good, until one of the disks died. The restore (reset-brick) took about a month, becaus

Re: [Gluster-users] To RAID or not to RAID...

Hi Markus, You are right .I think that the 3 node setup matches distributed volume. According to https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes/ Dispersed volumes use erasure codes to have a 'parity' on a separate brick. In such case you can afford to loos

Re: [Gluster-users] To RAID or not to RAID...

Hi Strahil, thanks for you answer - but now I am completely lost :) From this documentation: https://docs.oracle.com/cd/E52668_01/F10040/html/gluster-312-volume-distr-disp.html "As a dispersed volume must have a minimum of three bricks, a distributed dispersed volume must have at least six bri

Re: [Gluster-users] to RAID or not?

04.07.2016 19:01, Matt Robinson пишет: With mdadm any raid6 (especially with 12 disks) will be rubbish. Well, this can be offtopic, but could you, please, explain why? (never used md raid other than raid1... ) ___ Gluster-users mailing list Gluster

Re: [Gluster-users] to RAID or not?

If you go the ZFS route - be absolutely sure you set xattr=sa on all filesystems that will hold bricks BEFORE you create bricks on same. Not doing so will cause major problems with data that should be deleted not being reclaimed until after a forced dismount or reboot (which can take hours -> d

Re: [Gluster-users] to RAID or not?

On 5/07/2016 12:54 AM, Gandalf Corvotempesta wrote: No suggestions ? Il 14 giu 2016 10:01 AM, "Gandalf Corvotempesta" > ha scritto: Let's assume a small cluster made by 3 servers, 12 disks/bricks each. This cluster would be expanded to a maximu

Re: [Gluster-users] to RAID or not?

Agreed… It took me almost 2 years of tweaking and testing to get the performance I wanted. Different workloads require different configurations.Test different configurations and find what works best for you! > On Jul 4, 2016, at 2:15 PM, t...@encoding.com wrote: > > I would highly stres

Re: [Gluster-users] to RAID or not?

Sorry, example of 5 servers should read > server1 A & B replica to server 2 C & D > server2 A & B replica to server 3 C & D > server3 A & B replica to server 4 C & D > server4 A & B replica to server 5 C & D > server5 A & B replica to server 1 C & D Adding each server should be as simp

Re: [Gluster-users] to RAID or not?

I would highly stress, regardless of whatever solution you choose - make sure you test actual workload performance before going all-in. In my testing, performance (esp. iops and latency) decreased as I added bricks and additional nodes. Since you have many spindles now, I would encourage you t

Re: [Gluster-users] to RAID or not?

The fault tolerance is provided by Gluster replica translator. RAID0 to me is preferable to JBOD because you get 3x read performance and 3x write performance. If performance is not a concern, or if you only have 1GbE, then it may not matter, and you could just do JBOD with a ton of bricks. Th

Re: [Gluster-users] to RAID or not?

2016-07-04 19:44 GMT+02:00 Gandalf Corvotempesta : > So, any disk failure would me at least 6TB to be recovered via > network. This mean an high network utilization and as long gluster > doesn't have a dedicated network for replica, > this can slow down client access. Additionally, using a RAID-0

Re: [Gluster-users] to RAID or not?

2016-07-04 19:35 GMT+02:00 Russell Purinton : > For 3 servers with 12 disks each, I would do Hardware RAID0 (or madam if you > don’t have a RAID card) of 3 disks. So four 3-disk RAID0’s per server. 3 servers is just to start. We plan to use 5 server in shorter time and up to 15 on production. >

Re: [Gluster-users] to RAID or not?

For 3 servers with 12 disks each, I would do Hardware RAID0 (or madam if you don’t have a RAID card) of 3 disks. So four 3-disk RAID0’s per server. I would set them up as Replica 3 Arbiter 1 server1:/brickA server2:/brickC server3:/brickA server1:/brickB server2:/brickD server3:/brickB server2

Re: [Gluster-users] to RAID or not?

2016-07-04 19:25 GMT+02:00 Matt Robinson : > If you don't trust the hardware raid, then steer clear of raid-6 as mdadm > raid 6 is stupidly slow. > I don't completely trust hardware raid either, but rebuild times should be > under a day and in order to lose a raid-6 array you have to lose 3 disks

Re: [Gluster-users] to RAID or not?

If you don't trust the hardware raid, then steer clear of raid-6 as mdadm raid 6 is stupidly slow. I don't completely trust hardware raid either, but rebuild times should be under a day and in order to lose a raid-6 array you have to lose 3 disks. My own systems are hardware raid-6. If you're not

Re: [Gluster-users] to RAID or not?

IMHO you use raid for performance reasons and gluster for fault tolerance and scale. On July 4, 2016 7:54:44 AM PDT, Gandalf Corvotempesta wrote: >No suggestions ? >Il 14 giu 2016 10:01 AM, "Gandalf Corvotempesta" < >gandalf.corvotempe...@gmail.com> ha scritto: > >> Let's assume a small cluster

Re: [Gluster-users] to RAID or not?

Hi Gandalf Not suggesting really here but just mentioning what I am using: I am using an HBA adapter with 12 disks so basically JBOD but I am using ZFS and have an array of 12 disks in RAIDZ2 (sort of RAID6 but ZFS-style). I am pretty happy with that setup so far. CheersML On Monday, July

Re: [Gluster-users] to RAID or not?

Hi Gandalf, Are you using hardware raid or mdadm? On high quality hardware raid, a 12 disk raid-6 is pretty solid. With mdadm any raid6 (especially with 12 disks) will be rubbish. Matt. > On 4 Jul 2016, at 15:54:44, Gandalf Corvotempesta > wrote: > > No suggestions ? > > Il 14 giu 2016 10:

Re: [Gluster-users] to RAID or not?

No suggestions ? Il 14 giu 2016 10:01 AM, "Gandalf Corvotempesta" < gandalf.corvotempe...@gmail.com> ha scritto: > Let's assume a small cluster made by 3 servers, 12 disks/bricks each. > This cluster would be expanded to a maximum of 15 servers in near future. > > What do you suggest, a JBOD or a