Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi all, we have opened the VOTE thread a few weeks ago as we hoped that this DISCUSS thread exchange had been exhaustive. If so, we would you like any interested party to. cast a vote there. Of course we're happy to further progress the KIP discussion if needed. Thanks Edo & Adrian On Wed, 5

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Jorge! On Fri, 30 Jun 2023 at 15:47, Jorge Esteban Quilcate Otoya wrote: > > Thank you both for the replies! A couple more comments: > > The current proposal is to have ‘record.validation.policy’ per topic > (default null). A flag would be something like > ‘record.validation.policy.enable’

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Thank you both for the replies! A couple more comments: On Tue, 27 Jun 2023 at 14:57, Edoardo Comar wrote: > Hi Jorge > thanks for the feedback. Comments inline below > > > 1. Similar to Kirk's first point, I'm also concerned on how would the > > plugin developers / operators be able to apply

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Andrew, thanks for your comments ! Please see replies inline below. On Mon, 26 Jun 2023 at 16:51, Andrew Schofield wrote: > 4) For a new interface, I wonder whether it would be better to use > TopicIdPartition rather > than TopicPartition. Topic IDs are gradually spreading across the public

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Tom, thanks for tour comments, replies inline below. On Thu, 22 Jun 2023 at 10:58, Tom Bentley wrote: > > Hi Edorado and Adrian, > > Thanks for the KIP. > > I think it would be good to elaborate on exactly how validate() gets > called, because I think there are a number of potential

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Jorge thanks for the feedback. Comments inline below > 1. Similar to Kirk's first point, I'm also concerned on how would the > plugin developers / operators be able to apply multiple policies and how > configurations would be passed to each policy. We’ve only attempted to tackle the “one

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Edo, Thanks for the KIP. Looks like a useful improvement. I have some comments/questions. 4) For a new interface, I wonder whether it would be better to use TopicIdPartition rather than TopicPartition. Topic IDs are gradually spreading across the public interfaces for Kafka. 5) The new

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Kirk, thanks for your comments. > 1. Does record.validation.policy.class.name support multiple classes, or just > one? I’m probably not wrapping my head around it, > but I’d imagine different policies for different families or groupings of > topics, thus the need for supporting multiple

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Edorado and Adrian, Thanks for the KIP. I think it would be good to elaborate on exactly how validate() gets called, because I think there are a number of potential problems, or at least things to consider. >From the broker's point of view, validate() can do arbitrary things. It might never

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Eduardo, Adrian. Thanks for the KIP. I agree that allowing custom validations on the broker-side addresses a real gap as you clearly stated on the motivation. Some initial thoughts from my side: 1. Similar to Kirk's first point, I'm also concerned on how would the plugin developers /

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hi Edo/Adrian! Thanks for the KIP. I have some questions, and apologies that the may fall under the “stupid” column because I’m not that familiar with this area :) 1. Does record.validation.policy.class.name support multiple classes, or just one? I’m probably not wrapping my head around it,

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Thanks Николай, We’d like to open a vote next week. Hopefully getting some more feedback before then. Edo On Wed, 7 Jun 2023 at 19:15, Николай Ижиков wrote: > Hello. > > As author of one of related KIPs I’m +1 for this change. > Long waited feature. > > > 7 июня 2023 г., в 19:02, Edoardo

Re: [DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Hello. As author of one of related KIPs I’m +1 for this change. Long waited feature. > 7 июня 2023 г., в 19:02, Edoardo Comar написал(а): > > Dear all, > Adrian and I would like to start a discussion thread on > > KIP-940: Broker extension point for validating record contents at produce time

[DISCUSS] KIP-940: Broker extension point for validating record contents at produce time

Dear all, Adrian and I would like to start a discussion thread on KIP-940: Broker extension point for validating record contents at produce time https://cwiki.apache.org/confluence/display/KAFKA/KIP-940%3A+Broker+extension+point+for+validating+record+contents+at+produce+time This KIP proposes a