Hello George, Thanks for submitting this KIP. On the high-level I think I agree that handling keyed messages is a very complicated issue and maybe we can just start with the easy scenario that does not involve them. Pushing the burden to admin users to determine if it is really safe to delete partitions (i.e. there should be no key-ed messages, OR message keys are never used in the partitioner). Regarding the detailed proposal, I have some clarification questions / comments above:
1) Compatibility wise, we need to clarify when talking to the old versioned clients who do not recognize the added `status` field, what the brokers should return for those read-only / offline topics. My guess is that, for old versioned brokers we would not include the status field, but would exclude / include the partitions for producers / consumers accordingly. 2) In the upcoming release, with KIP-500/691 we will have a zookeeper-free quorum-based controller mechanism. So I'd suggest we "rebase" the proposal on top of that given the timing of this KIP, i.e. consider moving those zk-paths as for how the new controller could handle the requests. I'd recommend incorporating the proposal with KIP-691. 3) For the read-only partitions, who's responsible for finally switching the status of those partitions to offline when the retention period passed? I cannot tell for sure from the diagram in the KIP. Note that in practice that retention period could be days or even weeks. Also, could we still force delete a read-only partition before its retention period? 4) Another thing to consider is how deleting partitions work with adding partitions. For example, if a topic has 3 partitions with 0/1 online and 2 read-only, and a new admin request is received to add a new partition to that topic, how would this be handled? Guozhang On Tue, Dec 8, 2020 at 8:05 AM georgeshu(舒国强) <george...@tencent.com> wrote: > Hello, > > We write up a KIP based on a straightforward mechanism implemented and > tested in order to solve a practical issue in production. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-694%3A+Support+Reducing+Partitions+for+Topics > Look forward to hearing feedback and suggestions. > > Thanks! > > > -- -- Guozhang
