RAID has nothing to do with the overall availability of your system, it is just increasing the per node reliability.
Regards, Istvan On Wed, Oct 22, 2014 at 11:01 AM, Gwen Shapira <gshap...@cloudera.com> wrote: > RAID-10? > Interesting choice for a system where the data is already replicated > between nodes. Is it to avoid the cost of large replication over the > network? how large are these disks? > > On Wed, Oct 22, 2014 at 10:00 AM, Todd Palino <tpal...@gmail.com> wrote: > > In fact there are many more than 4000 open files. Many of our brokers run > > with 28,000+ open files (regular file handles, not network connections). > In > > our case, we're beefing up the disk performance as much as we can by > > running in a RAID-10 configuration with 14 disks. > > > > -Todd > > > > On Tue, Oct 21, 2014 at 7:58 PM, Xiaobin She <xiaobin...@gmail.com> > wrote: > > > >> Todd, > >> > >> Actually I'm wondering how kafka handle so much partition, with one > >> partition there is at least one file on disk, and with 4000 partition, > >> there will be at least 4000 files. > >> > >> When all these partitions have write request, how did Kafka make the > write > >> operation on the disk to be sequential (which is emphasized in the > design > >> document of Kafka) and make sure the disk access is effective? > >> > >> Thank you for your reply. > >> > >> xiaobinshe > >> > >> > >> > >> 2014-10-22 5:10 GMT+08:00 Todd Palino <tpal...@gmail.com>: > >> > >> > As far as the number of partitions a single broker can handle, we've > set > >> > our cap at 4000 partitions (including replicas). Above that we've seen > >> some > >> > performance and stability issues. > >> > > >> > -Todd > >> > > >> > On Tue, Oct 21, 2014 at 12:15 AM, Xiaobin She <xiaobin...@gmail.com> > >> > wrote: > >> > > >> > > hello, everyone > >> > > > >> > > I'm new to kafka, I'm wondering what's the max num of partition can > one > >> > > siggle machine handle in Kafka? > >> > > > >> > > Is there an sugeest num? > >> > > > >> > > Thanks. > >> > > > >> > > xiaobinshe > >> > > > >> > > >> > -- the sun shines for all
