Thanks! Chris. One question : We will need to wait to fix all SMTs in the KIP and release it to user, or we can split some fixed SMTs to user early? (I noticed that the PR of Mario ready to merge and it effect only ExtractField and InsertField)
On 2024/06/13 17:01:49 Chris Egerton wrote: > Hi Punsak, > > If nobody has signaled their intent to contribute that work yet (which I > believe is the case), you are welcome to take it on yourself! > > Cheers, > > Cheers > > On Thu, Jun 13, 2024, 12:52 Punsak Incham <pu...@mfec.co.th.invalid> wrote: > > > My customers are using these SMTs, I think it can affect to their project > > (about data correctness) in the future. > > > > (You may think why I'm not developing it and send to my customers > > directly? Because they will use only SMTs that published by Kafka.) > > > > On 2024/06/13 16:38:45 Punsak Incham wrote: > > > Hi all, > > > I noticed that the PR of Mario effected only InsertField and > > ExtractField, so I'd like to amend others SMTs in that KIP because I have > > experienced to develop custom SMTs. > > > Can I join to contribute (open PR)? > > > > > > On 2024/06/13 15:59:36 Greg Harris wrote: > > > > Hey Mario, > > > > > > > > Thanks for updating the KIP. > > > > +1 (binding) > > > > > > > > Greg > > > > > > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale <mv...@redhat.com> > > > > wrote: > > > > > > > > > Hi Greg, > > > > > > > > > > > However, the KIP still omits the MaskField and ValueToKey > > > > > transformations. > > > > > This looks like just a typo to me, should we update the KIP before > > closing > > > > > the vote? > > > > > > > > > > Good catch. I have just updated the KIP. > > > > > > > > > > I think that we can close the voting and mark it as approved, right? > > > > > > > > > > Thank you all. > > > > > > > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris <gr...@aiven.io.invalid> > > > > > wrote: > > > > > > > > > > > Hi Mario, > > > > > > > > > > > > Thank you for mentioning the earlier DISCUSS thread. I found this > > comment > > > > > > from Chris, which was agreed upon and applied to the KIP: > > > > > > > > > > > > > Yes, I think we should just do one KIP for all the SMTs. You > > don't have > > > > > > to > > > > > > > implement everything all at once or by yourself, but I don't see > > why we > > > > > > > should require one or more follow-up KIPs to apply the exact same > > > > > changes > > > > > > > to the SMTs we missed the first time. > > > > > > > > > > > > However, the KIP still omits the MaskField and ValueToKey > > > > > transformations. > > > > > > This looks like just a typo to me, should we update the KIP before > > > > > closing > > > > > > the vote? > > > > > > > > > > > > Thanks, > > > > > > Greg > > > > > > > > > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale < > > mv...@redhat.com> > > > > > > wrote: > > > > > > > > > > > > > Hi Greg, > > > > > > > > > > > > > > The same check was made by Chris Egerton during the discussion > > thread. > > > > > > > > > > > > > > The original KIP scope was just for the InsertField, > > ExtractField SMTs, > > > > > > > then we decided to enlarge the scope of only the KIP to also > > other > > > > > > > potential affected SMTs. > > > > > > > > > > > > > > As of now the PR scope, instead, is only for InsertField and > > > > > ExtractField > > > > > > > SMTs. > > > > > > > > > > > > > > Thanks, > > > > > > > Mario. > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris > > > > > > <greg.har...@aiven.io.invalid > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Mario, > > > > > > > > > > > > > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted > > to > > > > > > verify a > > > > > > > > discrepancy I noticed. > > > > > > > > > > > > > > > > I checked all of the call-sites of Struct#get(Field) and > > > > > > > Struct#get(String) > > > > > > > > in Kafka, and noticed there are some call-sites which are not > > > > > included > > > > > > in > > > > > > > > the KIP. > > > > > > > > 1. The Flatten transformation seems to already have the > > > > > > > > "replace.null.with.default=false" behavior unconditionally. > > > > > > > > 2. The MaskField transformation unconditionally injects default > > > > > values > > > > > > > for > > > > > > > > top-level structs. > > > > > > > > 3. The ValueTokey transformation injects defaults for each of > > the > > > > > > > > configured "fields" > > > > > > > > 4. The Values#convertToString method injects default values, > > but as > > > > > > this > > > > > > > > isn't configurable I think we'll need to leave it as-is. > > > > > > > > > > > > > > > > Did you intend to add this configuration to the Flatten, > > MaskField, > > > > > or > > > > > > > > ValueToKey transformations, or should those be addressed in a > > > > > separate > > > > > > > > effort? > > > > > > > > Since the "false" behavior is desirable, I think the Flatten > > > > > transform > > > > > > > > could be left as-is, rather than adding a configuration that > > nobody > > > > > > > needs. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Greg > > > > > > > > > > > > > > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya <ya...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks for the KIP! > > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale < > > > > > > mvit...@redhat.com > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > Just wanted to bump up this thread for visibility. > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison < > > > > > > > > mickael.mai...@gmail.com > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Mario, > > > > > > > > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > Thanks for the KIP! > > > > > > > > > > > > > > > > > > > > > > Mickael > > > > > > > > > > > > > > > > > > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale < > > > > > > > > > mvit...@redhat.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > After 7 days I received only one vote. Should I > > suppose this > > > > > > will > > > > > > > > not > > > > > > > > > > be > > > > > > > > > > > > approved? > > > > > > > > > > > > > > > > > > > > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton > > > > > > > > > <chr...@aiven.io.invalid > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the KIP! +1 (binding) > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale < > > > > > > > > > > mvit...@redhat.com > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to call a vote on KIP-1040 which aims to > > improve > > > > > > > > > handling > > > > > > > > > > of > > > > > > > > > > > > > > nullable values in InsertField, ExtractField, and > > other > > > > > > > > > > > transformations > > > > > > > > > > > > > > > > > > > > > > > > > > > > KIP - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677 > > > > > > > > > > > > > > > > > > > > > > > > > > > > Discussion thread - > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks and regards, > > > > > > > > > > > > > > Mario > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > Mario Fiore Vitale > > > > > > > > > > > > > > > > > > > > > > > > Senior Software Engineer > > > > > > > > > > > > > > > > > > > > > > > > Red Hat <https://www.redhat.com/> > > > > > > > > > > > > <https://www.redhat.com/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Mario Fiore Vitale > > > > > > > > > > > > > > > > > > > > Senior Software Engineer > > > > > > > > > > > > > > > > > > > > Red Hat <https://www.redhat.com/> > > > > > > > > > > <https://www.redhat.com/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Mario Fiore Vitale > > > > > > > > > > > > > > Senior Software Engineer > > > > > > > > > > > > > > Red Hat <https://www.redhat.com/> > > > > > > > <https://www.redhat.com/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Mario Fiore Vitale > > > > > > > > > > Senior Software Engineer > > > > > > > > > > Red Hat <https://www.redhat.com/> > > > > > <https://www.redhat.com/> > > > > > > > > > > > > > > > Punsak Incham > > > Data Platform Engineer (Kafka and Confluent) > > > MFEC > > > > > > > Punsak Incham > > Data Platform Engineer (Kafka and Confluent) MFEC > > > Punsak Incham Data Platform Engineer (Kafka and Confluent) MFEC
