Re: Kafka with RAID 5 on. busy cluster.
RAID 5 typically is slower because Kafka is very write heavy load and that creates a bottleneck because writes to any disk require parity writes on the other disks. -hans > On Mar 28, 2020, at 2:55 PM, Vishal Santoshi > wrote: > > Ny one ? We doing a series of tests to be confident, but if there is some > data folks, who have had RAID 5 on kafka, have to share, please do. > > Regards. > >> On Mon, Mar 23, 2020 at 11:29 PM Vishal Santoshi >> wrote: >> >> << In RAID 5 one can loose more than only one disk RAID here will be data >> corruption. In RAID 5 if one looses more than only one disk RAID there will be data >> corruption. >> >> On Mon, Mar 23, 2020 at 11:27 PM Vishal Santoshi < >> vishal.santo...@gmail.com> wrote: >> >>> One obvious issue is disk failure toleration . As in if RF =3 on.normal >>> JBOD disk failure toleration is 2. In RAID 5 one can loose more than only >>> one disk RAID here will be data corruption. effectively making the broker >>> unusable, thus reducing our drive failure toleration to 2 drives ON 2 >>> different brokers with the added caveat that we loose the whole broker as >>> well ? >>> >>> >>> On Mon, Mar 23, 2020 at 10:42 PM Vishal Santoshi < >>> vishal.santo...@gmail.com> wrote: >>> We have a pretty busy kafka cluster with SSD and plain JBOD. We planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we know that there is a write overhead parity bit on blocks and recreating a dead drive ) and can share there experience on it . Confluent advises against it but there are obvious ease one gets with RAID ( RAID 10 is to expensive space wise ) Any advise /comments etc will be highly appreciated. Regards.
Re: Kafka with RAID 5 on. busy cluster.
Ny one ? We doing a series of tests to be confident, but if there is some data folks, who have had RAID 5 on kafka, have to share, please do. Regards. On Mon, Mar 23, 2020 at 11:29 PM Vishal Santoshi wrote: > << In RAID 5 one can loose more than only one disk RAID here will be data > corruption. > >> In RAID 5 if one looses more than only one disk RAID there will be data > corruption. > > On Mon, Mar 23, 2020 at 11:27 PM Vishal Santoshi < > vishal.santo...@gmail.com> wrote: > >> One obvious issue is disk failure toleration . As in if RF =3 on.normal >> JBOD disk failure toleration is 2. In RAID 5 one can loose more than only >> one disk RAID here will be data corruption. effectively making the broker >> unusable, thus reducing our drive failure toleration to 2 drives ON 2 >> different brokers with the added caveat that we loose the whole broker as >> well ? >> >> >> On Mon, Mar 23, 2020 at 10:42 PM Vishal Santoshi < >> vishal.santo...@gmail.com> wrote: >> >>> We have a pretty busy kafka cluster with SSD and plain JBOD. We >>> planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD >>> bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we >>> know that there is a write overhead parity bit on blocks and recreating a >>> dead drive ) and can share there experience on it . Confluent advises >>> against it but there are obvious ease one gets with RAID ( RAID 10 is to >>> expensive space wise ) Any advise /comments etc will be highly >>> appreciated. >>> >>> Regards. >>> >>>
Re: Kafka with RAID 5 on. busy cluster.
<< In RAID 5 one can loose more than only one disk RAID here will be data corruption. >> In RAID 5 if one looses more than only one disk RAID there will be data corruption. On Mon, Mar 23, 2020 at 11:27 PM Vishal Santoshi wrote: > One obvious issue is disk failure toleration . As in if RF =3 on.normal > JBOD disk failure toleration is 2. In RAID 5 one can loose more than only > one disk RAID here will be data corruption. effectively making the broker > unusable, thus reducing our drive failure toleration to 2 drives ON 2 > different brokers with the added caveat that we loose the whole broker as > well ? > > > On Mon, Mar 23, 2020 at 10:42 PM Vishal Santoshi < > vishal.santo...@gmail.com> wrote: > >> We have a pretty busy kafka cluster with SSD and plain JBOD. We >> planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD >> bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we >> know that there is a write overhead parity bit on blocks and recreating a >> dead drive ) and can share there experience on it . Confluent advises >> against it but there are obvious ease one gets with RAID ( RAID 10 is to >> expensive space wise ) Any advise /comments etc will be highly >> appreciated. >> >> Regards. >> >>
Re: Kafka with RAID 5 on. busy cluster.
One obvious issue is disk failure toleration . As in if RF =3 on.normal JBOD disk failure toleration is 2. In RAID 5 one can loose more than only one disk RAID here will be data corruption. effectively making the broker unusable, thus reducing our drive failure toleration to 2 drives ON 2 different brokers with the added caveat that we loose the whole broker as well ? On Mon, Mar 23, 2020 at 10:42 PM Vishal Santoshi wrote: > We have a pretty busy kafka cluster with SSD and plain JBOD. We > planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD > bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we > know that there is a write overhead parity bit on blocks and recreating a > dead drive ) and can share there experience on it . Confluent advises > against it but there are obvious ease one gets with RAID ( RAID 10 is to > expensive space wise ) Any advise /comments etc will be highly > appreciated. > > Regards. > >
Kafka with RAID 5 on. busy cluster.
We have a pretty busy kafka cluster with SSD and plain JBOD. We planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we know that there is a write overhead parity bit on blocks and recreating a dead drive ) and can share there experience on it . Confluent advises against it but there are obvious ease one gets with RAID ( RAID 10 is to expensive space wise ) Any advise /comments etc will be highly appreciated. Regards.
