Hi all, I have a few questions. If you include all the related SMTs in the KIP, such as MaskField and ValueToKey, As a user, when will I be able to use the updated SMTs? Do all the related SMTs in the KIP need to be fixed first before you release them for users like me to use?
Punsak Incham Data Platform Engineer (Kafka and Confluent) MFEC -----Original Message----- From: Greg Harris <greg.har...@aiven.io.INVALID> Sent: 13 มิถุนายน 2567 0:14 To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations Hi Mario, Thank you for mentioning the earlier DISCUSS thread. I found this comment from Chris, which was agreed upon and applied to the KIP: > Yes, I think we should just do one KIP for all the SMTs. You don't > have to implement everything all at once or by yourself, but I don't > see why we should require one or more follow-up KIPs to apply the > exact same changes to the SMTs we missed the first time. However, the KIP still omits the MaskField and ValueToKey transformations. This looks like just a typo to me, should we update the KIP before closing the vote? Thanks, Greg On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <mvit...@redhat.com> wrote: > Hi Greg, > > The same check was made by Chris Egerton during the discussion thread. > > The original KIP scope was just for the InsertField, ExtractField > SMTs, then we decided to enlarge the scope of only the KIP to also > other potential affected SMTs. > > As of now the PR scope, instead, is only for InsertField and > ExtractField SMTs. > > Thanks, > Mario. > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris > <greg.har...@aiven.io.invalid > > > wrote: > > > Hi Mario, > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted to > > verify a discrepancy I noticed. > > > > I checked all of the call-sites of Struct#get(Field) and > Struct#get(String) > > in Kafka, and noticed there are some call-sites which are not > > included in the KIP. > > 1. The Flatten transformation seems to already have the > > "replace.null.with.default=false" behavior unconditionally. > > 2. The MaskField transformation unconditionally injects default > > values > for > > top-level structs. > > 3. The ValueTokey transformation injects defaults for each of the > > configured "fields" > > 4. The Values#convertToString method injects default values, but as > > this isn't configurable I think we'll need to leave it as-is. > > > > Did you intend to add this configuration to the Flatten, MaskField, > > or ValueToKey transformations, or should those be addressed in a > > separate effort? > > Since the "false" behavior is desirable, I think the Flatten > > transform could be left as-is, rather than adding a configuration > > that nobody > needs. > > > > Thanks, > > Greg > > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya <yash.ma...@gmail.com> wrote: > > > > > Thanks for the KIP! > > > > > > +1 (binding) > > > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale > > > <mvit...@redhat.com > > > > > wrote: > > > > > > > Hi all, > > > > > > > > Just wanted to bump up this thread for visibility. > > > > > > > > Thanks! > > > > > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison < > > mickael.mai...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hi Mario, > > > > > > > > > > +1 (binding) > > > > > Thanks for the KIP! > > > > > > > > > > Mickael > > > > > > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale < > > > mvit...@redhat.com> > > > > > wrote: > > > > > > > > > > > > After 7 days I received only one vote. Should I suppose this > > > > > > will > > not > > > > be > > > > > > approved? > > > > > > > > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton > > > <chr...@aiven.io.invalid > > > > > > > > > > > wrote: > > > > > > > > > > > > > Thanks for the KIP! +1 (binding) > > > > > > > > > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale < > > > > mvit...@redhat.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > I'd like to call a vote on KIP-1040 which aims to > > > > > > > > improve > > > handling > > > > of > > > > > > > > nullable values in InsertField, ExtractField, and other > > > > > transformations > > > > > > > > > > > > > > > > KIP - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik > i.apache.org%2Fconfluence%2Fpages%2Fviewpage.action%3FpageId%3D3037946 > 77&data=05%7C02%7Cpunsak%40mfec.co.th%7C36a25ef5a026433cbcc808dc8b0323 > c8%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C638538092832326790%7CU > nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha > WwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=AWg1i6RQZdMsijQ0Ew0Tr3sj7bDYUsfi% > 2B%2BVxyg%2BT2wo%3D&reserved=0 > > > > > > > > > > > > > > > > Discussion thread - > > > > > > > > > > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli > > sts.apache.org%2Fthread%2Fggqqqjbg6ccpz8g6ztyj7oxr80q5184n&data=05%7 > > C02%7Cpunsak%40mfec.co.th%7C36a25ef5a026433cbcc808dc8b0323c8%7C74105 > > ed972ff4685915475f7408b6f67%7C1%7C0%7C638538092832336686%7CUnknown%7 > > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ > > XVCI6Mn0%3D%7C0%7C%7C%7C&sdata=HJqhd5EckaAHm%2FdGNR2kDz32ujjtO10GzDQ > > 5zRUfpoo%3D&reserved=0 > > > > > > > > > > > > > > > > Thanks and regards, > > > > > > > > Mario > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Mario Fiore Vitale > > > > > > > > > > > > Senior Software Engineer > > > > > > > > > > > > Red Hat > > > > > > <https://apc01.safelinks.protection.outlook.com/?url=https%3 > > > > > > A%2F%2Fwww.redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th% > > > > > > 7C36a25ef5a026433cbcc808dc8b0323c8%7C74105ed972ff4685915475f > > > > > > 7408b6f67%7C1%7C0%7C638538092832339791%7CUnknown%7CTWFpbGZsb > > > > > > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > > > > > > CI6Mn0%3D%7C0%7C%7C%7C&sdata=SKkHkLVmWcuuNOEoAxJJLR9AapWKZk5 > > > > > > 85EpKCq3a2UY%3D&reserved=0> > > > > > > <https://apc01.safelinks.protection.outlook.com/?url=https%3 > > > > > > A%2F%2Fwww.redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th% > > > > > > 7C36a25ef5a026433cbcc808dc8b0323c8%7C74105ed972ff4685915475f > > > > > > 7408b6f67%7C1%7C0%7C638538092832342688%7CUnknown%7CTWFpbGZsb > > > > > > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > > > > > > CI6Mn0%3D%7C0%7C%7C%7C&sdata=v500tcqgR%2BkU%2BQwNrr6F1C9z8Vx > > > > > > SIaV9Ezmya4zba7w%3D&reserved=0> > > > > > > > > > > > > > > > > > > -- > > > > > > > > Mario Fiore Vitale > > > > > > > > Senior Software Engineer > > > > > > > > Red Hat > > > > <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F > > > > %2Fwww.redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th%7C36a25e > > > > f5a026433cbcc808dc8b0323c8%7C74105ed972ff4685915475f7408b6f67%7C > > > > 1%7C0%7C638538092832345604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wL > > > > jAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C% > > > > 7C&sdata=oIYFovw105h4EFg9XeXXEKRtBri1lIPdZjk%2B3neTaUM%3D&reserv > > > > ed=0> > > > > <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F > > > > %2Fwww.redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th%7C36a25e > > > > f5a026433cbcc808dc8b0323c8%7C74105ed972ff4685915475f7408b6f67%7C > > > > 1%7C0%7C638538092832348501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wL > > > > jAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C% > > > > 7C&sdata=92PKFhFsNwiwHH0Kpe5u%2B7%2BuM46nNLWc4CmtKkVYLo0%3D&rese > > > > rved=0> > > > > > > > > > > > > -- > > Mario Fiore Vitale > > Senior Software Engineer > > Red Hat > <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww > .redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th%7C36a25ef5a026433cbc > c808dc8b0323c8%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C6385380928 > 32351300%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL > CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=1LT6Qi%2FFDregYV8j8XX > O4vvJ1JBeSUDjZMFjOAhItJQ%3D&reserved=0> > <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww > .redhat.com%2F&data=05%7C02%7Cpunsak%40mfec.co.th%7C36a25ef5a026433cbc > c808dc8b0323c8%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C6385380928 > 32354202%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL > CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=NnXIdCllrhDeI1u%2F4fM > nvHzCVoA3mIVeN%2BmLkbOxv9o%3D&reserved=0> >
