For the znodes created under /admin/reassignments/, would it be better if the partition znodes for the same topic are put under subtree headed by topic name ? In Tom's example, instead of:
/admin/reassignments/mytopic-42 the znode would be: /admin/reassignments/mytopic/42 Cheers On Fri, Dec 8, 2017 at 5:41 PM, Jun Rao <j...@confluent.io> wrote: > Hi, Tom, > > Thanks for the KIP. It definitely addresses one of the pain points in > partition reassignment. Another issue that it also addresses is the ZK node > size limit when writing the reassignment JSON. > > My only concern is that the KIP needs to create one watcher per reassigned > partition. This could add overhead in ZK and complexity for debugging when > lots of partitions are being reassigned simultaneously. We could > potentially improve this by introducing a separate ZK path for change > notification as we do for configs. For example, every time we change the > assignment for a set of partitions, we could further write a sequential > node /admin/reassignment_changes/[change_x]. That way, the controller only > needs to watch the change path. Once a change is triggered, the controller > can read everything under /admin/reassignments/. > > Jun > > > On Wed, Dec 6, 2017 at 1:19 PM, Tom Bentley <t.j.bent...@gmail.com> wrote: > > > Hi, > > > > This is still very new, but I wanted some quick feedback on a preliminary > > KIP which could, I think, help with providing an AdminClient API for > > partition reassignment. > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 236%3A+Interruptible+Partition+Reassignment > > > > I wasn't sure whether to start fleshing out a whole AdminClient API in > this > > KIP (which would make it very big, and difficult to read), or whether to > > break it down into smaller KIPs (which makes it easier to read and > > implement in pieces, but harder to get a high-level picture of the > ultimate > > destination). For now I've gone for a very small initial KIP, but I'm > happy > > to sketch the bigger picture here if people are interested. > > > > Cheers, > > > > Tom > > >
