Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-12-04 Thread Chris Egerton
Hi all, It seems like there are no objections to this KIP, so I've kicked off a vote thread: https://lists.apache.org/thread/dgq332o5j25rwddbvfydf7ttrclldw17 Cheers, Chris On Fri, Nov 24, 2023 at 10:39 PM Chris Egerton wrote: > Hi Yash, > > Thanks for taking a look! Yeah, it looks like we'll

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-24 Thread Chris Egerton
Hi Yash, Thanks for taking a look! Yeah, it looks like we'll be forced to hold onto the property until 5.0 if we can't introduce it at least one minor release before 4.0. I don't think this is the worst thing. Although it'd be nice to have the property completely removed when designing features

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-22 Thread Yash Mayya
Hi Chris, Thanks for the well written and comprehensive KIP! Given that we're already past the KIP freeze deadline for 3.7.0 ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.7.0) and there may not be a 3.8.0 release before the 4.0.0 release, would we then be forced to punt the

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-20 Thread Chris Egerton
Hi Hector, Thanks for taking a look! I think the key difference between the proposed behavior and the rejected alternative is that the set of tasks that will be running with the former is still a complete set of tasks, whereas the set of tasks in the latter is a subset of tasks. Also noteworthy

Re:[DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-20 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Thanks for the KIP Chris, adding this check makes total sense. I do have one question. The second paragraph in the Public Interfaces section states: "If the connector generated excessive tasks after being reconfigured, then any existing tasks for the connector will be allowed to continue