Another question is on the compatibility. Since now there are 2 ways of specifying a partition reassignment, one under /admin/reassign_partitions and the other under /admin/reassignments, we probably want to prevent the same topic being reassigned under both paths at the same time? Thanks,
Jun On Fri, Dec 8, 2017 at 5:41 PM, Jun Rao <j...@confluent.io> wrote: > Hi, Tom, > > Thanks for the KIP. It definitely addresses one of the pain points in > partition reassignment. Another issue that it also addresses is the ZK node > size limit when writing the reassignment JSON. > > My only concern is that the KIP needs to create one watcher per reassigned > partition. This could add overhead in ZK and complexity for debugging when > lots of partitions are being reassigned simultaneously. We could > potentially improve this by introducing a separate ZK path for change > notification as we do for configs. For example, every time we change the > assignment for a set of partitions, we could further write a sequential > node /admin/reassignment_changes/[change_x]. That way, the controller > only needs to watch the change path. Once a change is triggered, the > controller can read everything under /admin/reassignments/. > > Jun > > > On Wed, Dec 6, 2017 at 1:19 PM, Tom Bentley <t.j.bent...@gmail.com> wrote: > >> Hi, >> >> This is still very new, but I wanted some quick feedback on a preliminary >> KIP which could, I think, help with providing an AdminClient API for >> partition reassignment. >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-236% >> 3A+Interruptible+Partition+Reassignment >> >> I wasn't sure whether to start fleshing out a whole AdminClient API in >> this >> KIP (which would make it very big, and difficult to read), or whether to >> break it down into smaller KIPs (which makes it easier to read and >> implement in pieces, but harder to get a high-level picture of the >> ultimate >> destination). For now I've gone for a very small initial KIP, but I'm >> happy >> to sketch the bigger picture here if people are interested. >> >> Cheers, >> >> Tom >> > >
