Hey Jason, Thanks for the KIP! I have some questions below.
The KIP defines "UnderSynchronized" as partitions which have an in-sync replica set that is smaller than the topic's replication factor. Can you clarify how we decide the topic's replication factor? Currently the possible sources include the topic config znode, replica list in the topic znode or the broker default config. The topic config znode will have replication factor only if the replication factor differs from the default broker config when the topic is created, right? Replica list in the topic znode will expand during reassignment which is not useful for the purpose of this KIP. The default replication factor in the broker may be changed after the topic is created. If user increases default replication factor from 3 to 4, would all topics be considered as `UnderSynchronized`? The KIP proposes to use over-replicated partitions to track the progress of a reassignment. Currently ReassignPartitionsCommand tracks the progress of reassignment by looking at the reassignment znode. Is there concern with still using this approach to track the progress of reassignment? Thanks, Dong On Fri, Aug 3, 2018 at 11:23 AM, Jason Gustafson <ja...@confluent.io> wrote: > Hey Ismael, > > Yeah, my initial inclination was to redefine URP as well. My only doubt was > how it would affect existing tools which might depend on URPs to track the > progress of a reassignment. I decided to be conservative in the end, but > I'd reconsider if we think it is not a major concern. It is annoying to > need a new category. > > About your question about storage in ZK, I can't think of anything > additional that we need. Probably the main difficulty is getting access to > the replication factor in the topic utility. My basic thought was just to > collect the URPs (as we know them today) and use the config API to > partition them based on the replication factor. Do you see any problems > with this? > > -Jason > > > On Thu, Aug 2, 2018 at 12:14 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Thanks Jason. This is definitely a pain point. I actually prefer the > option > > to redefine what under-replicated means (currently under rejected > > alternatives). Also, do we need to make changes to what we store in ZK? > If > > so, that should be in the KIP too. > > > > Ismael > > > > On Thu, Aug 2, 2018 at 11:45 AM Jason Gustafson <ja...@confluent.io> > > wrote: > > > > > Hey All, > > > > > > Another day, another KIP. This one is hopefully straightforward: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-352% > > 3A+Distinguish+URPs+caused+by+reassignment > > > . > > > Have a look and let me know what you think! > > > > > > Thanks, > > > Jason > > > > > >
