Re: Kafka partitions unbalanced

2015-06-02 Thread Vijay Patil
I ran into similar issue. I configured 3 disks, but partitions were allocated only to 2 disks (disk2 and disk3). Then I found that the left out disk (disk1) was already hosting lot number of other partitions from different topics. So may be partition allocation happens based on how many partitions

Re: Kafka partitions unbalanced

2015-06-01 Thread Jason Rosenberg
Andrew Otto, This is a known problem (and which I have run into as well). Generally, my solution has been to increase the number of partitions such that the granularity of partitions is much higher than the number of disks, such that its more unlikely for the imbalance to be significant. I

RE: Kafka partitions unbalanced

2015-05-27 Thread Jonathan Creasy
I have a similar issue, let me know how it goes. :) -Original Message- From: Andrew Otto [mailto:ao...@wikimedia.org] Sent: Wednesday, May 27, 2015 3:12 PM To: users@kafka.apache.org Subject: Kafka partitions unbalanced Hi all, I’ve recently noticed that our broker log.dirs are using