LInkedin's cruise-control https://github.com/linkedin/cruise-control
has numerous goals, including disk, network, cpu, rack awareness, leadership distribution etc. You can have separate disk/network limits per broker (ours are all the same fwiw) We use it and it does a stellar job of keeping a cluster balanced. On Tue, Oct 9, 2018 at 8:11 PM Vincent Bernardi <[email protected]> wrote: > Hello everyone, > I couldn't find an answer to this on the web so: > If I have a Kafka cluster where nodes have different disk size, is there a > way to have an automatic partitioning aware of these disk sizes (i.e. > allocating 4 partitions to the 4TB node and 1 partition to the 1TB node)? I > know I can partition by hand, but it would be much easier if I could just > declare a max disk space to the partitioner one way or the other. > > Thanks in advance, > Vincent Bernardi >
