Thanks Todd. Let us know when these tools are announced. On Sun, May 24, 2015 at 11:00 PM, Todd Palino <tpal...@gmail.com> wrote:
> See, this is why I should never say anything :) > > The version I have right now is very limited - it only does a clone (we > needed it for some hardware testing) and a leader balance (does it using > partition reassignment without actually moving partitions). We have some > scripts that the other SREs have written that need to be incorporated which > do partition balancing by count and by size. > > Let me figure out where we are going to host this, but we can probably put > out the script we have in the next couple weeks. Then we can just iterate > on it. > > -Todd > > > On May 24, 2015, at 8:46 PM, Henry Cai <h...@pinterest.com.INVALID> > wrote: > > > > Todd, > > > > This is very promising. Do you know when will we be able to see your > tools > > released to public? > > > >> On Sun, May 24, 2015 at 7:54 PM, Todd Palino <tpal...@gmail.com> wrote: > >> > >> We've built tools on top of it that both build the list based on less > >> information (like "clone this broker to that one") and break it down > into a > >> configurable number of discrete moves so it doesn't tank the cluster. > >> > >> And yes, I've finally started the process of departing them from the > >> LinkedIn-specific tooling so we can release them to everyone else :) > >> > >> -Todd > >> > >>>> On May 24, 2015, at 7:45 PM, Henry Cai <h...@pinterest.com.INVALID> > >>> wrote: > >>> > >>> We have a kafka cluster with 10 brokers and we are using the kafka > >>> replication tool (kafka-reassign-partitions.sh) when we need to add > more > >>> brokers to the cluster. But this tool tends to move too many > >>> topic/partitions around at the same time which causes instability. Do > we > >>> have an option to do it more slowly (e.g. move one topic/partition at a > >>> step) or did some one build a tool on top of > >> 'kafka-reassign-partitions.sh'? > >>> > >>> Another use case is when a broker node went down, do we have a tool to > >> move > >>> the topic/partitions serviced by this node to the remaining nodes (and > >>> doing that in a fashion which doesn't cause too much instability)? > >> >
