RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-17 Thread Punsak Incham 
Hi all,

As @Mario Fiore Vitale mentioned, we have now 
received enough votes.

Therefore, should we close this voting and start the development?





On 2024/06/14 08:00:45 Mario Fiore Vitale wrote:

> Hi All,

>

> We have 4 binding +1 votes. Do you think the vote can be closed and

> mark the KIP as approved?

>

> Thanks,

> Mario.

>

> On Fri, Jun 14, 2024 at 8:44 AM Punsak Incham 
> mailto:pu...@mfec.co.th>> wrote:

>

> > Thanks! Chris.

> > Waiting @Mario Fiore Vitale to close the voting and mark it as approved.

> >

> > Cheers.

> >

> > On 2024/06/14 06:10:15 Chris Egerton wrote:

> > > We don't have to patch every SMT in the same release, we can

> > > definitely move incrementally. We'll just have to note in the

> > > release notes and the KIPs page (

> > > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2F

> > > cwik%2F=05%7C02%7Cpunsak%40mfec.co.th%7Cf933b95ace7c4eca8a7c0

> > > 8dc8ec20be4%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C638542211

> > > 281036334%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu

> > > MzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=u8Gak9%2F7IZ

> > > b7vxi2aAW26jup4IcLiyHXWPikv6gpSDo%3D=0

> > > i.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKafka%2BImprovement%

> > > 2BPr

> > > oposals=05%7C02%7Cpunsak%40mfec.co.th%7Ca24883ed513b45d31ca50

> > > 8dc8

> > > c3c2de1%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C6385394373000

> > > 4293

> > > 1%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB

> > > TiI6

> > > Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=mcPQROjQyuK1Z2vDl%2BLVki

> > > 8AeB

> > > FsnTIkqxaX8FR2LRA%3D=0) when the change for the KIP was

> > > applied to each SMT.

> > >

> > > On Thu, Jun 13, 2024 at 11:03 PM Punsak Incham

> > > mailto:pu...@mfec.co.th.invalid>>

> > > wrote:

> > >

> > > > Thanks! Chris.

> > > >

> > > > One question : We will need to wait to fix all SMTs in the KIP

> > > > and release it to user, or we can split some fixed SMTs to user early?

> > > > (I noticed that the PR of Mario ready to merge and it effect

> > > > only ExtractField and

> > > > InsertField)

> > > >

> > > >

> > > > On 2024/06/13 17:01:49 Chris Egerton wrote:

> > > > > Hi Punsak,

> > > > >

> > > > > If nobody has signaled their intent to contribute that work

> > > > > yet (which I believe is the case), you are welcome to take it

> > > > > on

> > yourself!

> > > > >

> > > > > Cheers,

> > > > >

> > > > > Cheers

> > > > >

> > > > > On Thu, Jun 13, 2024, 12:52 Punsak Incham

> > > > > mailto:pu...@mfec.co.th.invalid>>

> > > > wrote:

> > > > >

> > > > > > My customers are using these SMTs, I think it can affect to

> > > > > > their

> > > > project

> > > > > > (about data correctness) in the future.

> > > > > >

> > > > > > (You may think why I'm not developing it and send to my

> > > > > > customers directly? Because they will use only SMTs that

> > > > > > published by Kafka.)

> > > > > >

> > > > > > On 2024/06/13 16:38:45 Punsak Incham wrote:

> > > > > > > Hi all,

> > > > > > > I noticed that the PR of Mario effected only InsertField

> > > > > > > and

> > > > > > ExtractField, so I'd like to amend others SMTs in that KIP

> > > > > > because I

> > > > have

> > > > > > experienced to develop custom SMTs.

> > > > > > > Can I join to contribute (open PR)?

> > > > > > >

> > > > > > > On 2024/06/13 15:59:36 Greg Harris wrote:

> > > > > > > > Hey Mario,

> > > > > > > >

> > > > > > > > Thanks for updating the KIP.

> > > > > > > > +1 (binding)

> > > > > > > >

> > > > > > > > Greg

> > > > > > > >

> > > > > > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale <

> > > > mv...@redhat.com>

> > > > > > > > wrote:

> > > > > > > >

> > > > > > > > > Hi Greg,

> > > > > > > > >

> > > > > > > > > > However, the KIP still omits the MaskField and

> > > > > > > > > > ValueToKey

> > > > > > > > > transformations.

> > > > > > > > > This looks like just a typo to me, should we update

> > > > > > > > > the KIP

> > > > before

> > > > > > closing

> > > > > > > > > the vote?

> > > > > > > > >

> > > > > > > > > Good catch. I have just updated the KIP.

> > > > > > > > >

> > > > > > > > > I think that we can close the voting and mark it as

> > > > > > > > > approved,

> > > > right?

> > > > > > > > >

> > > > > > > > > Thank you all.

> > > > > > > > >

> > > > > > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris

> > > > mailto:gr...@aiven.io.invalid>>

> > > > > > > > > wrote:

> > > > > > > > >

> > > > > > > > > > Hi Mario,

> > > > > > > > > >

> > > > > > > > > > Thank you for mentioning the earlier DISCUSS thread.

> > > > > > > > > > I found

> > > > this

> > > > > > comment

> > > > > > > > > > from Chris, which was agreed upon and applied to the KIP:

> > > > > > > > > >

> > > > > > > > > > > Yes, I think we should just do one KIP for all the

> > > > > > > > > > > SMTs. You

> > > > > > don't have

> > > > > > > > > >

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-14 Thread Mario Fiore Vitale 
Hi All,

We have 4 binding +1 votes. Do you think the vote can be closed and mark
the KIP as approved?

Thanks,
Mario.

On Fri, Jun 14, 2024 at 8:44 AM Punsak Incham  wrote:

> Thanks! Chris.
> Waiting @Mario Fiore Vitale to close the voting and mark it as approved.
>
> Cheers.
>
> On 2024/06/14 06:10:15 Chris Egerton wrote:
> > We don't have to patch every SMT in the same release, we can
> > definitely move incrementally. We'll just have to note in the release
> > notes and the KIPs page (
> > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> > i.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKafka%2BImprovement%2BPr
> > oposals=05%7C02%7Cpunsak%40mfec.co.th%7Ca24883ed513b45d31ca508dc8
> > c3c2de1%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C63853943730004293
> > 1%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6
> > Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=mcPQROjQyuK1Z2vDl%2BLVki8AeB
> > FsnTIkqxaX8FR2LRA%3D=0) when the change for the KIP was
> > applied to each SMT.
> >
> > On Thu, Jun 13, 2024 at 11:03 PM Punsak Incham
> > 
> > wrote:
> >
> > > Thanks! Chris.
> > >
> > > One question : We will need to wait to fix all SMTs in the KIP and
> > > release it to user, or we can split some fixed SMTs to user early?
> > > (I noticed that the PR of Mario ready to merge and it effect only
> > > ExtractField and
> > > InsertField)
> > >
> > >
> > > On 2024/06/13 17:01:49 Chris Egerton wrote:
> > > > Hi Punsak,
> > > >
> > > > If nobody has signaled their intent to contribute that work yet
> > > > (which I believe is the case), you are welcome to take it on
> yourself!
> > > >
> > > > Cheers,
> > > >
> > > > Cheers
> > > >
> > > > On Thu, Jun 13, 2024, 12:52 Punsak Incham
> > > > 
> > > wrote:
> > > >
> > > > > My customers are using these SMTs, I think it can affect to
> > > > > their
> > > project
> > > > > (about data correctness) in the future.
> > > > >
> > > > > (You may think why I'm not developing it and send to my
> > > > > customers directly? Because they will use only SMTs that
> > > > > published by Kafka.)
> > > > >
> > > > > On 2024/06/13 16:38:45 Punsak Incham wrote:
> > > > > > Hi all,
> > > > > > I noticed that the PR of Mario effected only InsertField and
> > > > > ExtractField, so I'd like to amend others SMTs in that KIP
> > > > > because I
> > > have
> > > > > experienced to develop custom SMTs.
> > > > > > Can I join to contribute (open PR)?
> > > > > >
> > > > > > On 2024/06/13 15:59:36 Greg Harris wrote:
> > > > > > > Hey Mario,
> > > > > > >
> > > > > > > Thanks for updating the KIP.
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Greg
> > > > > > >
> > > > > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale <
> > > mv...@redhat.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Greg,
> > > > > > > >
> > > > > > > > > However, the KIP still omits the MaskField and
> > > > > > > > > ValueToKey
> > > > > > > > transformations.
> > > > > > > > This looks like just a typo to me, should we update the
> > > > > > > > KIP
> > > before
> > > > > closing
> > > > > > > > the vote?
> > > > > > > >
> > > > > > > > Good catch. I have just updated the KIP.
> > > > > > > >
> > > > > > > > I think that we can close the voting and mark it as
> > > > > > > > approved,
> > > right?
> > > > > > > >
> > > > > > > > Thank you all.
> > > > > > > >
> > > > > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris
> > > 
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Mario,
> > > > > > > > >
> > > > > > > > > Thank you for mentioning the earlier DISCUSS thread. I
> > > > > > > > > found
> > > this
> > > > > comment
> > > > > > > > > from Chris, which was agreed upon and applied to the KIP:
> > > > > > > > >
> > > > > > > > > > Yes, I think we should just do one KIP for all the
> > > > > > > > > > SMTs. You
> > > > > don't have
> > > > > > > > > to
> > > > > > > > > > implement everything all at once or by yourself, but I
> > > > > > > > > > don't
> > > see
> > > > > why we
> > > > > > > > > > should require one or more follow-up KIPs to apply the
> > > > > > > > > > exact
> > > same
> > > > > > > > changes
> > > > > > > > > > to the SMTs we missed the first time.
> > > > > > > > >
> > > > > > > > > However, the KIP still omits the MaskField and
> > > > > > > > > ValueToKey
> > > > > > > > transformations.
> > > > > > > > > This looks like just a typo to me, should we update the
> > > > > > > > > KIP
> > > before
> > > > > > > > closing
> > > > > > > > > the vote?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Greg
> > > > > > > > >
> > > > > > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <
> > > > > mv...@redhat.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Greg,
> > > > > > > > > >
> > > > > > > > > > The same check was made by Chris Egerton during the
> > > discussion
> > > > > thread.
> > > > > > > > > >
> > > > > > > > > > The original KIP scope was just for the InsertField,
> > > > >

RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-14 Thread Punsak Incham 
Thanks! Chris.
Waiting @Mario Fiore Vitale to close the voting and mark it as approved.

Cheers.

On 2024/06/14 06:10:15 Chris Egerton wrote:
> We don't have to patch every SMT in the same release, we can 
> definitely move incrementally. We'll just have to note in the release 
> notes and the KIPs page (
> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> i.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKafka%2BImprovement%2BPr
> oposals=05%7C02%7Cpunsak%40mfec.co.th%7Ca24883ed513b45d31ca508dc8
> c3c2de1%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C63853943730004293
> 1%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6
> Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=mcPQROjQyuK1Z2vDl%2BLVki8AeB
> FsnTIkqxaX8FR2LRA%3D=0) when the change for the KIP was 
> applied to each SMT.
>
> On Thu, Jun 13, 2024 at 11:03 PM Punsak Incham 
> 
> wrote:
>
> > Thanks! Chris.
> >
> > One question : We will need to wait to fix all SMTs in the KIP and 
> > release it to user, or we can split some fixed SMTs to user early? 
> > (I noticed that the PR of Mario ready to merge and it effect only 
> > ExtractField and
> > InsertField)
> >
> >
> > On 2024/06/13 17:01:49 Chris Egerton wrote:
> > > Hi Punsak,
> > >
> > > If nobody has signaled their intent to contribute that work yet 
> > > (which I believe is the case), you are welcome to take it on yourself!
> > >
> > > Cheers,
> > >
> > > Cheers
> > >
> > > On Thu, Jun 13, 2024, 12:52 Punsak Incham 
> > > 
> > wrote:
> > >
> > > > My customers are using these SMTs, I think it can affect to 
> > > > their
> > project
> > > > (about data correctness) in the future.
> > > >
> > > > (You may think why I'm not developing it and send to my 
> > > > customers directly? Because they will use only SMTs that 
> > > > published by Kafka.)
> > > >
> > > > On 2024/06/13 16:38:45 Punsak Incham wrote:
> > > > > Hi all,
> > > > > I noticed that the PR of Mario effected only InsertField and
> > > > ExtractField, so I'd like to amend others SMTs in that KIP 
> > > > because I
> > have
> > > > experienced to develop custom SMTs.
> > > > > Can I join to contribute (open PR)?
> > > > >
> > > > > On 2024/06/13 15:59:36 Greg Harris wrote:
> > > > > > Hey Mario,
> > > > > >
> > > > > > Thanks for updating the KIP.
> > > > > > +1 (binding)
> > > > > >
> > > > > > Greg
> > > > > >
> > > > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale <
> > mv...@redhat.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Greg,
> > > > > > >
> > > > > > > > However, the KIP still omits the MaskField and 
> > > > > > > > ValueToKey
> > > > > > > transformations.
> > > > > > > This looks like just a typo to me, should we update the 
> > > > > > > KIP
> > before
> > > > closing
> > > > > > > the vote?
> > > > > > >
> > > > > > > Good catch. I have just updated the KIP.
> > > > > > >
> > > > > > > I think that we can close the voting and mark it as 
> > > > > > > approved,
> > right?
> > > > > > >
> > > > > > > Thank you all.
> > > > > > >
> > > > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris
> > 
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Mario,
> > > > > > > >
> > > > > > > > Thank you for mentioning the earlier DISCUSS thread. I 
> > > > > > > > found
> > this
> > > > comment
> > > > > > > > from Chris, which was agreed upon and applied to the KIP:
> > > > > > > >
> > > > > > > > > Yes, I think we should just do one KIP for all the 
> > > > > > > > > SMTs. You
> > > > don't have
> > > > > > > > to
> > > > > > > > > implement everything all at once or by yourself, but I 
> > > > > > > > > don't
> > see
> > > > why we
> > > > > > > > > should require one or more follow-up KIPs to apply the 
> > > > > > > > > exact
> > same
> > > > > > > changes
> > > > > > > > > to the SMTs we missed the first time.
> > > > > > > >
> > > > > > > > However, the KIP still omits the MaskField and 
> > > > > > > > ValueToKey
> > > > > > > transformations.
> > > > > > > > This looks like just a typo to me, should we update the 
> > > > > > > > KIP
> > before
> > > > > > > closing
> > > > > > > > the vote?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Greg
> > > > > > > >
> > > > > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <
> > > > mv...@redhat.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Greg,
> > > > > > > > >
> > > > > > > > > The same check was made by Chris Egerton during the
> > discussion
> > > > thread.
> > > > > > > > >
> > > > > > > > > The original KIP scope was just for the InsertField,
> > > > ExtractField SMTs,
> > > > > > > > > then we decided to enlarge the scope of only the KIP 
> > > > > > > > > to also
> > > > other
> > > > > > > > > potential affected SMTs.
> > > > > > > > >
> > > > > > > > > As of now the PR scope, instead, is only for 
> > > > > > > > > InsertField and
> > > > > > > ExtractField
> > > > > > > > > SMTs.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Mario.
> > > > > > > > >
> > > > > > > > >
> > >

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-14 Thread Chris Egerton 
We don't have to patch every SMT in the same release, we can definitely
move incrementally. We'll just have to note in the release notes and the
KIPs page (
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals)
when the change for the KIP was applied to each SMT.

On Thu, Jun 13, 2024 at 11:03 PM Punsak Incham 
wrote:

> Thanks! Chris.
>
> One question : We will need to wait to fix all SMTs in the KIP and release
> it to user, or we can split some fixed SMTs to user early? (I noticed that
> the PR of Mario ready to merge and it effect only ExtractField and
> InsertField)
>
>
> On 2024/06/13 17:01:49 Chris Egerton wrote:
> > Hi Punsak,
> >
> > If nobody has signaled their intent to contribute that work yet (which I
> > believe is the case), you are welcome to take it on yourself!
> >
> > Cheers,
> >
> > Cheers
> >
> > On Thu, Jun 13, 2024, 12:52 Punsak Incham 
> wrote:
> >
> > > My customers are using these SMTs, I think it can affect to their
> project
> > > (about data correctness) in the future.
> > >
> > > (You may think why I'm not developing it and send to my customers
> > > directly? Because they will use only SMTs that published by Kafka.)
> > >
> > > On 2024/06/13 16:38:45 Punsak Incham wrote:
> > > > Hi all,
> > > > I noticed that the PR of Mario effected only InsertField and
> > > ExtractField, so I'd like to amend others SMTs in that KIP because I
> have
> > > experienced to develop custom SMTs.
> > > > Can I join to contribute (open PR)?
> > > >
> > > > On 2024/06/13 15:59:36 Greg Harris wrote:
> > > > > Hey Mario,
> > > > >
> > > > > Thanks for updating the KIP.
> > > > > +1 (binding)
> > > > >
> > > > > Greg
> > > > >
> > > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale <
> mv...@redhat.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Greg,
> > > > > >
> > > > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > > > transformations.
> > > > > > This looks like just a typo to me, should we update the KIP
> before
> > > closing
> > > > > > the vote?
> > > > > >
> > > > > > Good catch. I have just updated the KIP.
> > > > > >
> > > > > > I think that we can close the voting and mark it as approved,
> right?
> > > > > >
> > > > > > Thank you all.
> > > > > >
> > > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris
> 
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Mario,
> > > > > > >
> > > > > > > Thank you for mentioning the earlier DISCUSS thread. I found
> this
> > > comment
> > > > > > > from Chris, which was agreed upon and applied to the KIP:
> > > > > > >
> > > > > > > > Yes, I think we should just do one KIP for all the SMTs. You
> > > don't have
> > > > > > > to
> > > > > > > > implement everything all at once or by yourself, but I don't
> see
> > > why we
> > > > > > > > should require one or more follow-up KIPs to apply the exact
> same
> > > > > > changes
> > > > > > > > to the SMTs we missed the first time.
> > > > > > >
> > > > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > > > transformations.
> > > > > > > This looks like just a typo to me, should we update the KIP
> before
> > > > > > closing
> > > > > > > the vote?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Greg
> > > > > > >
> > > > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <
> > > mv...@redhat.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Greg,
> > > > > > > >
> > > > > > > > The same check was made by Chris Egerton during the
> discussion
> > > thread.
> > > > > > > >
> > > > > > > > The original KIP scope was just for the InsertField,
> > > ExtractField SMTs,
> > > > > > > > then we decided to enlarge the scope of only the KIP to also
> > > other
> > > > > > > > potential affected SMTs.
> > > > > > > >
> > > > > > > > As of now the PR scope, instead, is only for InsertField and
> > > > > > ExtractField
> > > > > > > > SMTs.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mario.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> > > > > > >  > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Mario,
> > > > > > > > >
> > > > > > > > > Thanks for the KIP. I'm on-board with this KIP, I just
> wanted
> > > to
> > > > > > > verify a
> > > > > > > > > discrepancy I noticed.
> > > > > > > > >
> > > > > > > > > I checked all of the call-sites of Struct#get(Field) and
> > > > > > > > Struct#get(String)
> > > > > > > > > in Kafka, and noticed there are some call-sites which are
> not
> > > > > > included
> > > > > > > in
> > > > > > > > > the KIP.
> > > > > > > > > 1. The Flatten transformation seems to already have the
> > > > > > > > > "replace.null.with.default=false" behavior unconditionally.
> > > > > > > > > 2. The MaskField transformation unconditionally injects
> default
> > > > > > values
> > > > > > > > for
> > > > > > > > > top-level structs.
> > > > > > > > > 3. The ValueTokey transformation injects defaults for each
> of
> > > the
> > > >

RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Punsak Incham 
Thanks! Chris.

One question : We will need to wait to fix all SMTs in the KIP and release it 
to user, or we can split some fixed SMTs to user early? (I noticed that the PR 
of Mario ready to merge and it effect only ExtractField and InsertField)


On 2024/06/13 17:01:49 Chris Egerton wrote:
> Hi Punsak,
>
> If nobody has signaled their intent to contribute that work yet (which I
> believe is the case), you are welcome to take it on yourself!
>
> Cheers,
>
> Cheers
>
> On Thu, Jun 13, 2024, 12:52 Punsak Incham  wrote:
>
> > My customers are using these SMTs, I think it can affect to their project
> > (about data correctness) in the future.
> >
> > (You may think why I'm not developing it and send to my customers
> > directly? Because they will use only SMTs that published by Kafka.)
> >
> > On 2024/06/13 16:38:45 Punsak Incham wrote:
> > > Hi all,
> > > I noticed that the PR of Mario effected only InsertField and
> > ExtractField, so I'd like to amend others SMTs in that KIP because I have
> > experienced to develop custom SMTs.
> > > Can I join to contribute (open PR)?
> > >
> > > On 2024/06/13 15:59:36 Greg Harris wrote:
> > > > Hey Mario,
> > > >
> > > > Thanks for updating the KIP.
> > > > +1 (binding)
> > > >
> > > > Greg
> > > >
> > > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale 
> > > > wrote:
> > > >
> > > > > Hi Greg,
> > > > >
> > > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > > transformations.
> > > > > This looks like just a typo to me, should we update the KIP before
> > closing
> > > > > the vote?
> > > > >
> > > > > Good catch. I have just updated the KIP.
> > > > >
> > > > > I think that we can close the voting and mark it as approved, right?
> > > > >
> > > > > Thank you all.
> > > > >
> > > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
> > > > > wrote:
> > > > >
> > > > > > Hi Mario,
> > > > > >
> > > > > > Thank you for mentioning the earlier DISCUSS thread. I found this
> > comment
> > > > > > from Chris, which was agreed upon and applied to the KIP:
> > > > > >
> > > > > > > Yes, I think we should just do one KIP for all the SMTs. You
> > don't have
> > > > > > to
> > > > > > > implement everything all at once or by yourself, but I don't see
> > why we
> > > > > > > should require one or more follow-up KIPs to apply the exact same
> > > > > changes
> > > > > > > to the SMTs we missed the first time.
> > > > > >
> > > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > > transformations.
> > > > > > This looks like just a typo to me, should we update the KIP before
> > > > > closing
> > > > > > the vote?
> > > > > >
> > > > > > Thanks,
> > > > > > Greg
> > > > > >
> > > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <
> > mv...@redhat.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Greg,
> > > > > > >
> > > > > > > The same check was made by Chris Egerton during the discussion
> > thread.
> > > > > > >
> > > > > > > The original KIP scope was just for the InsertField,
> > ExtractField SMTs,
> > > > > > > then we decided to enlarge the scope of only the KIP to also
> > other
> > > > > > > potential affected SMTs.
> > > > > > >
> > > > > > > As of now the PR scope, instead, is only for InsertField and
> > > > > ExtractField
> > > > > > > SMTs.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mario.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> > > > > >  > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Mario,
> > > > > > > >
> > > > > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted
> > to
> > > > > > verify a
> > > > > > > > discrepancy I noticed.
> > > > > > > >
> > > > > > > > I checked all of the call-sites of Struct#get(Field) and
> > > > > > > Struct#get(String)
> > > > > > > > in Kafka, and noticed there are some call-sites which are not
> > > > > included
> > > > > > in
> > > > > > > > the KIP.
> > > > > > > > 1. The Flatten transformation seems to already have the
> > > > > > > > "replace.null.with.default=false" behavior unconditionally.
> > > > > > > > 2. The MaskField transformation unconditionally injects default
> > > > > values
> > > > > > > for
> > > > > > > > top-level structs.
> > > > > > > > 3. The ValueTokey transformation injects defaults for each of
> > the
> > > > > > > > configured "fields"
> > > > > > > > 4. The Values#convertToString method injects default values,
> > but as
> > > > > > this
> > > > > > > > isn't configurable I think we'll need to leave it as-is.
> > > > > > > >
> > > > > > > > Did you intend to add this configuration to the Flatten,
> > MaskField,
> > > > > or
> > > > > > > > ValueToKey transformations, or should those be addressed in a
> > > > > separate
> > > > > > > > effort?
> > > > > > > > Since the "false" behavior is desirable, I think the Flatten
> > > > > transform
> > > > > > > > could be left as-is, rather than adding a configuration that
> > nobody
> > > > > > > needs.
> > > > >

RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Punsak Incham 
My customers are using these SMTs, I think it can affect to their project 
(about data correctness) in the future.

(You may think why I'm not developing it and send to my customers directly? 
Because they will use only SMTs that published by Kafka.)

On 2024/06/13 16:38:45 Punsak Incham wrote:
> Hi all,
> I noticed that the PR of Mario effected only InsertField and ExtractField, so 
> I'd like to amend others SMTs in that KIP because I have experienced to 
> develop custom SMTs.
> Can I join to contribute (open PR)?
>
> On 2024/06/13 15:59:36 Greg Harris wrote:
> > Hey Mario,
> >
> > Thanks for updating the KIP.
> > +1 (binding)
> >
> > Greg
> >
> > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale 
> > wrote:
> >
> > > Hi Greg,
> > >
> > > > However, the KIP still omits the MaskField and ValueToKey
> > > transformations.
> > > This looks like just a typo to me, should we update the KIP before closing
> > > the vote?
> > >
> > > Good catch. I have just updated the KIP.
> > >
> > > I think that we can close the voting and mark it as approved, right?
> > >
> > > Thank you all.
> > >
> > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
> > > wrote:
> > >
> > > > Hi Mario,
> > > >
> > > > Thank you for mentioning the earlier DISCUSS thread. I found this 
> > > > comment
> > > > from Chris, which was agreed upon and applied to the KIP:
> > > >
> > > > > Yes, I think we should just do one KIP for all the SMTs. You don't 
> > > > > have
> > > > to
> > > > > implement everything all at once or by yourself, but I don't see why 
> > > > > we
> > > > > should require one or more follow-up KIPs to apply the exact same
> > > changes
> > > > > to the SMTs we missed the first time.
> > > >
> > > > However, the KIP still omits the MaskField and ValueToKey
> > > transformations.
> > > > This looks like just a typo to me, should we update the KIP before
> > > closing
> > > > the vote?
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
> > > > wrote:
> > > >
> > > > > Hi Greg,
> > > > >
> > > > > The same check was made by Chris Egerton during the discussion thread.
> > > > >
> > > > > The original KIP scope was just for the InsertField, ExtractField 
> > > > > SMTs,
> > > > > then we decided to enlarge the scope of only the KIP to also other
> > > > > potential affected SMTs.
> > > > >
> > > > > As of now the PR scope, instead, is only for InsertField and
> > > ExtractField
> > > > > SMTs.
> > > > >
> > > > > Thanks,
> > > > > Mario.
> > > > >
> > > > >
> > > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> > > >  > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mario,
> > > > > >
> > > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted to
> > > > verify a
> > > > > > discrepancy I noticed.
> > > > > >
> > > > > > I checked all of the call-sites of Struct#get(Field) and
> > > > > Struct#get(String)
> > > > > > in Kafka, and noticed there are some call-sites which are not
> > > included
> > > > in
> > > > > > the KIP.
> > > > > > 1. The Flatten transformation seems to already have the
> > > > > > "replace.null.with.default=false" behavior unconditionally.
> > > > > > 2. The MaskField transformation unconditionally injects default
> > > values
> > > > > for
> > > > > > top-level structs.
> > > > > > 3. The ValueTokey transformation injects defaults for each of the
> > > > > > configured "fields"
> > > > > > 4. The Values#convertToString method injects default values, but as
> > > > this
> > > > > > isn't configurable I think we'll need to leave it as-is.
> > > > > >
> > > > > > Did you intend to add this configuration to the Flatten, MaskField,
> > > or
> > > > > > ValueToKey transformations, or should those be addressed in a
> > > separate
> > > > > > effort?
> > > > > > Since the "false" behavior is desirable, I think the Flatten
> > > transform
> > > > > > could be left as-is, rather than adding a configuration that nobody
> > > > > needs.
> > > > > >
> > > > > > Thanks,
> > > > > > Greg
> > > > > >
> > > > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya 
> > > > wrote:
> > > > > >
> > > > > > > Thanks for the KIP!
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale <
> > > > mvit...@redhat.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Just wanted to bump up this thread for visibility.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > > > > > mickael.mai...@gmail.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Mario,
> > > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > > > Thanks for the KIP!
> > > > > > > > >
> > > > > > > > > Mickael
> > > > > > > > >
> > > > > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > > > > > mvit...@redhat.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > >

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Chris Egerton 
Hi Punsak,

If nobody has signaled their intent to contribute that work yet (which I
believe is the case), you are welcome to take it on yourself!

Cheers,

Cheers

On Thu, Jun 13, 2024, 12:52 Punsak Incham  wrote:

> My customers are using these SMTs, I think it can affect to their project
> (about data correctness) in the future.
>
> (You may think why I'm not developing it and send to my customers
> directly? Because they will use only SMTs that published by Kafka.)
>
> On 2024/06/13 16:38:45 Punsak Incham wrote:
> > Hi all,
> > I noticed that the PR of Mario effected only InsertField and
> ExtractField, so I'd like to amend others SMTs in that KIP because I have
> experienced to develop custom SMTs.
> > Can I join to contribute (open PR)?
> >
> > On 2024/06/13 15:59:36 Greg Harris wrote:
> > > Hey Mario,
> > >
> > > Thanks for updating the KIP.
> > > +1 (binding)
> > >
> > > Greg
> > >
> > > On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale 
> > > wrote:
> > >
> > > > Hi Greg,
> > > >
> > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > transformations.
> > > > This looks like just a typo to me, should we update the KIP before
> closing
> > > > the vote?
> > > >
> > > > Good catch. I have just updated the KIP.
> > > >
> > > > I think that we can close the voting and mark it as approved, right?
> > > >
> > > > Thank you all.
> > > >
> > > > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
> > > > wrote:
> > > >
> > > > > Hi Mario,
> > > > >
> > > > > Thank you for mentioning the earlier DISCUSS thread. I found this
> comment
> > > > > from Chris, which was agreed upon and applied to the KIP:
> > > > >
> > > > > > Yes, I think we should just do one KIP for all the SMTs. You
> don't have
> > > > > to
> > > > > > implement everything all at once or by yourself, but I don't see
> why we
> > > > > > should require one or more follow-up KIPs to apply the exact same
> > > > changes
> > > > > > to the SMTs we missed the first time.
> > > > >
> > > > > However, the KIP still omits the MaskField and ValueToKey
> > > > transformations.
> > > > > This looks like just a typo to me, should we update the KIP before
> > > > closing
> > > > > the vote?
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale <
> mv...@redhat.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Greg,
> > > > > >
> > > > > > The same check was made by Chris Egerton during the discussion
> thread.
> > > > > >
> > > > > > The original KIP scope was just for the InsertField,
> ExtractField SMTs,
> > > > > > then we decided to enlarge the scope of only the KIP to also
> other
> > > > > > potential affected SMTs.
> > > > > >
> > > > > > As of now the PR scope, instead, is only for InsertField and
> > > > ExtractField
> > > > > > SMTs.
> > > > > >
> > > > > > Thanks,
> > > > > > Mario.
> > > > > >
> > > > > >
> > > > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> > > > >  > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Mario,
> > > > > > >
> > > > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted
> to
> > > > > verify a
> > > > > > > discrepancy I noticed.
> > > > > > >
> > > > > > > I checked all of the call-sites of Struct#get(Field) and
> > > > > > Struct#get(String)
> > > > > > > in Kafka, and noticed there are some call-sites which are not
> > > > included
> > > > > in
> > > > > > > the KIP.
> > > > > > > 1. The Flatten transformation seems to already have the
> > > > > > > "replace.null.with.default=false" behavior unconditionally.
> > > > > > > 2. The MaskField transformation unconditionally injects default
> > > > values
> > > > > > for
> > > > > > > top-level structs.
> > > > > > > 3. The ValueTokey transformation injects defaults for each of
> the
> > > > > > > configured "fields"
> > > > > > > 4. The Values#convertToString method injects default values,
> but as
> > > > > this
> > > > > > > isn't configurable I think we'll need to leave it as-is.
> > > > > > >
> > > > > > > Did you intend to add this configuration to the Flatten,
> MaskField,
> > > > or
> > > > > > > ValueToKey transformations, or should those be addressed in a
> > > > separate
> > > > > > > effort?
> > > > > > > Since the "false" behavior is desirable, I think the Flatten
> > > > transform
> > > > > > > could be left as-is, rather than adding a configuration that
> nobody
> > > > > > needs.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Greg
> > > > > > >
> > > > > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya 
> > > > > wrote:
> > > > > > >
> > > > > > > > Thanks for the KIP!
> > > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale <
> > > > > mvit...@redhat.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Just wanted to bump up this thread for visibility.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
>

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Greg Harris 
Hey Mario,

Thanks for updating the KIP.
+1 (binding)

Greg

On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale 
wrote:

> Hi Greg,
>
> > However, the KIP still omits the MaskField and ValueToKey
> transformations.
> This looks like just a typo to me, should we update the KIP before closing
> the vote?
>
> Good catch. I have just updated the KIP.
>
> I think that we can close the voting and mark it as approved, right?
>
> Thank you all.
>
> On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
> wrote:
>
> > Hi Mario,
> >
> > Thank you for mentioning the earlier DISCUSS thread. I found this comment
> > from Chris, which was agreed upon and applied to the KIP:
> >
> > > Yes, I think we should just do one KIP for all the SMTs. You don't have
> > to
> > > implement everything all at once or by yourself, but I don't see why we
> > > should require one or more follow-up KIPs to apply the exact same
> changes
> > > to the SMTs we missed the first time.
> >
> > However, the KIP still omits the MaskField and ValueToKey
> transformations.
> > This looks like just a typo to me, should we update the KIP before
> closing
> > the vote?
> >
> > Thanks,
> > Greg
> >
> > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
> > wrote:
> >
> > > Hi Greg,
> > >
> > > The same check was made by Chris Egerton during the discussion thread.
> > >
> > > The original KIP scope was just for the InsertField, ExtractField SMTs,
> > > then we decided to enlarge the scope of only the KIP to also other
> > > potential affected SMTs.
> > >
> > > As of now the PR scope, instead, is only for InsertField and
> ExtractField
> > > SMTs.
> > >
> > > Thanks,
> > > Mario.
> > >
> > >
> > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> >  > > >
> > > wrote:
> > >
> > > > Hi Mario,
> > > >
> > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted to
> > verify a
> > > > discrepancy I noticed.
> > > >
> > > > I checked all of the call-sites of Struct#get(Field) and
> > > Struct#get(String)
> > > > in Kafka, and noticed there are some call-sites which are not
> included
> > in
> > > > the KIP.
> > > > 1. The Flatten transformation seems to already have the
> > > > "replace.null.with.default=false" behavior unconditionally.
> > > > 2. The MaskField transformation unconditionally injects default
> values
> > > for
> > > > top-level structs.
> > > > 3. The ValueTokey transformation injects defaults for each of the
> > > > configured "fields"
> > > > 4. The Values#convertToString method injects default values, but as
> > this
> > > > isn't configurable I think we'll need to leave it as-is.
> > > >
> > > > Did you intend to add this configuration to the Flatten, MaskField,
> or
> > > > ValueToKey transformations, or should those be addressed in a
> separate
> > > > effort?
> > > > Since the "false" behavior is desirable, I think the Flatten
> transform
> > > > could be left as-is, rather than adding a configuration that nobody
> > > needs.
> > > >
> > > > Thanks,
> > > > Greg
> > > >
> > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya 
> > wrote:
> > > >
> > > > > Thanks for the KIP!
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale <
> > mvit...@redhat.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Just wanted to bump up this thread for visibility.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > > > mickael.mai...@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Mario,
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > > Thanks for the KIP!
> > > > > > >
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > > > mvit...@redhat.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > After 7 days I received only one vote. Should I suppose this
> > will
> > > > not
> > > > > > be
> > > > > > > > approved?
> > > > > > > >
> > > > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> > > > >  > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks for the KIP! +1 (binding)
> > > > > > > > >
> > > > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > > > > mvit...@redhat.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > I'd like to call a vote on KIP-1040 which aims to improve
> > > > > handling
> > > > > > of
> > > > > > > > > > nullable values in InsertField, ExtractField, and other
> > > > > > > transformations
> > > > > > > > > >
> > > > > > > > > > KIP -
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > > > > > >
> > > > > > > > > > Discussion thread -
> > > > > > > > > >
> > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > > > > > >
> > > > > >

RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Punsak Incham 
Hi all,
I noticed that the PR of Mario effected only InsertField and ExtractField, so 
I'd like to amend others SMTs in that KIP because I have experienced to develop 
custom SMTs. 
Can I join to contribute (open PR)?

On 2024/06/13 15:59:36 Greg Harris wrote:
> Hey Mario,
>
> Thanks for updating the KIP.
> +1 (binding)
>
> Greg
>
> On Thu, Jun 13, 2024 at 1:32 AM Mario Fiore Vitale 
> wrote:
>
> > Hi Greg,
> >
> > > However, the KIP still omits the MaskField and ValueToKey
> > transformations.
> > This looks like just a typo to me, should we update the KIP before closing
> > the vote?
> >
> > Good catch. I have just updated the KIP.
> >
> > I think that we can close the voting and mark it as approved, right?
> >
> > Thank you all.
> >
> > On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
> > wrote:
> >
> > > Hi Mario,
> > >
> > > Thank you for mentioning the earlier DISCUSS thread. I found this comment
> > > from Chris, which was agreed upon and applied to the KIP:
> > >
> > > > Yes, I think we should just do one KIP for all the SMTs. You don't have
> > > to
> > > > implement everything all at once or by yourself, but I don't see why we
> > > > should require one or more follow-up KIPs to apply the exact same
> > changes
> > > > to the SMTs we missed the first time.
> > >
> > > However, the KIP still omits the MaskField and ValueToKey
> > transformations.
> > > This looks like just a typo to me, should we update the KIP before
> > closing
> > > the vote?
> > >
> > > Thanks,
> > > Greg
> > >
> > > On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
> > > wrote:
> > >
> > > > Hi Greg,
> > > >
> > > > The same check was made by Chris Egerton during the discussion thread.
> > > >
> > > > The original KIP scope was just for the InsertField, ExtractField SMTs,
> > > > then we decided to enlarge the scope of only the KIP to also other
> > > > potential affected SMTs.
> > > >
> > > > As of now the PR scope, instead, is only for InsertField and
> > ExtractField
> > > > SMTs.
> > > >
> > > > Thanks,
> > > > Mario.
> > > >
> > > >
> > > > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
> > >  > > > >
> > > > wrote:
> > > >
> > > > > Hi Mario,
> > > > >
> > > > > Thanks for the KIP. I'm on-board with this KIP, I just wanted to
> > > verify a
> > > > > discrepancy I noticed.
> > > > >
> > > > > I checked all of the call-sites of Struct#get(Field) and
> > > > Struct#get(String)
> > > > > in Kafka, and noticed there are some call-sites which are not
> > included
> > > in
> > > > > the KIP.
> > > > > 1. The Flatten transformation seems to already have the
> > > > > "replace.null.with.default=false" behavior unconditionally.
> > > > > 2. The MaskField transformation unconditionally injects default
> > values
> > > > for
> > > > > top-level structs.
> > > > > 3. The ValueTokey transformation injects defaults for each of the
> > > > > configured "fields"
> > > > > 4. The Values#convertToString method injects default values, but as
> > > this
> > > > > isn't configurable I think we'll need to leave it as-is.
> > > > >
> > > > > Did you intend to add this configuration to the Flatten, MaskField,
> > or
> > > > > ValueToKey transformations, or should those be addressed in a
> > separate
> > > > > effort?
> > > > > Since the "false" behavior is desirable, I think the Flatten
> > transform
> > > > > could be left as-is, rather than adding a configuration that nobody
> > > > needs.
> > > > >
> > > > > Thanks,
> > > > > Greg
> > > > >
> > > > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya 
> > > wrote:
> > > > >
> > > > > > Thanks for the KIP!
> > > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale <
> > > mvit...@redhat.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Just wanted to bump up this thread for visibility.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > > > > mickael.mai...@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Mario,
> > > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > > Thanks for the KIP!
> > > > > > > >
> > > > > > > > Mickael
> > > > > > > >
> > > > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > > > > mvit...@redhat.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > After 7 days I received only one vote. Should I suppose this
> > > will
> > > > > not
> > > > > > > be
> > > > > > > > > approved?
> > > > > > > > >
> > > > > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> > > > > >  > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for the KIP! +1 (binding)
> > > > > > > > > >
> > > > > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > > > > > mvit...@redhat.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > > > > >
> > > > > > > > > > > I'd like to

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-13 Thread Mario Fiore Vitale 
Hi Greg,

> However, the KIP still omits the MaskField and ValueToKey transformations.
This looks like just a typo to me, should we update the KIP before closing
the vote?

Good catch. I have just updated the KIP.

I think that we can close the voting and mark it as approved, right?

Thank you all.

On Wed, Jun 12, 2024 at 7:21 PM Greg Harris 
wrote:

> Hi Mario,
>
> Thank you for mentioning the earlier DISCUSS thread. I found this comment
> from Chris, which was agreed upon and applied to the KIP:
>
> > Yes, I think we should just do one KIP for all the SMTs. You don't have
> to
> > implement everything all at once or by yourself, but I don't see why we
> > should require one or more follow-up KIPs to apply the exact same changes
> > to the SMTs we missed the first time.
>
> However, the KIP still omits the MaskField and ValueToKey transformations.
> This looks like just a typo to me, should we update the KIP before closing
> the vote?
>
> Thanks,
> Greg
>
> On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
> wrote:
>
> > Hi Greg,
> >
> > The same check was made by Chris Egerton during the discussion thread.
> >
> > The original KIP scope was just for the InsertField, ExtractField SMTs,
> > then we decided to enlarge the scope of only the KIP to also other
> > potential affected SMTs.
> >
> > As of now the PR scope, instead, is only for InsertField and ExtractField
> > SMTs.
> >
> > Thanks,
> > Mario.
> >
> >
> > On Wed, Jun 12, 2024 at 12:08 AM Greg Harris
>  > >
> > wrote:
> >
> > > Hi Mario,
> > >
> > > Thanks for the KIP. I'm on-board with this KIP, I just wanted to
> verify a
> > > discrepancy I noticed.
> > >
> > > I checked all of the call-sites of Struct#get(Field) and
> > Struct#get(String)
> > > in Kafka, and noticed there are some call-sites which are not included
> in
> > > the KIP.
> > > 1. The Flatten transformation seems to already have the
> > > "replace.null.with.default=false" behavior unconditionally.
> > > 2. The MaskField transformation unconditionally injects default values
> > for
> > > top-level structs.
> > > 3. The ValueTokey transformation injects defaults for each of the
> > > configured "fields"
> > > 4. The Values#convertToString method injects default values, but as
> this
> > > isn't configurable I think we'll need to leave it as-is.
> > >
> > > Did you intend to add this configuration to the Flatten, MaskField, or
> > > ValueToKey transformations, or should those be addressed in a separate
> > > effort?
> > > Since the "false" behavior is desirable, I think the Flatten transform
> > > could be left as-is, rather than adding a configuration that nobody
> > needs.
> > >
> > > Thanks,
> > > Greg
> > >
> > > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya 
> wrote:
> > >
> > > > Thanks for the KIP!
> > > >
> > > > +1 (binding)
> > > >
> > > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale <
> mvit...@redhat.com
> > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Just wanted to bump up this thread for visibility.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > > mickael.mai...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mario,
> > > > > >
> > > > > > +1 (binding)
> > > > > > Thanks for the KIP!
> > > > > >
> > > > > > Mickael
> > > > > >
> > > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > > mvit...@redhat.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > After 7 days I received only one vote. Should I suppose this
> will
> > > not
> > > > > be
> > > > > > > approved?
> > > > > > >
> > > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> > > >  > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks for the KIP! +1 (binding)
> > > > > > > >
> > > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > > > mvit...@redhat.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi everyone,
> > > > > > > > >
> > > > > > > > > I'd like to call a vote on KIP-1040 which aims to improve
> > > > handling
> > > > > of
> > > > > > > > > nullable values in InsertField, ExtractField, and other
> > > > > > transformations
> > > > > > > > >
> > > > > > > > > KIP -
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > > > > >
> > > > > > > > > Discussion thread -
> > > > > > > > >
> > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > > > > >
> > > > > > > > > Thanks and regards,
> > > > > > > > > Mario
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Mario Fiore Vitale
> > > > > > >
> > > > > > > Senior Software Engineer
> > > > > > >
> > > > > > > Red Hat 
> > > > > > > 
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Mario Fiore Vitale
> > > > >
> > > > > Senior Software Engineer

RE: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-12 Thread Punsak Incham 
Hi all,
I have a few questions.

If you include all the related SMTs in the KIP, such as MaskField and 
ValueToKey,  As a user, when will I be able to use the updated SMTs? Do all the 
related SMTs in the KIP need to be fixed first before you release them for 
users like me to use?


Punsak Incham
Data Platform Engineer (Kafka and Confluent)
MFEC


-Original Message-
From: Greg Harris  
Sent: 13 มิถุนายน 2567 0:14
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-1040: Improve handling of nullable values in 
InsertField, ExtractField, and other transformations

Hi Mario,

Thank you for mentioning the earlier DISCUSS thread. I found this comment from 
Chris, which was agreed upon and applied to the KIP:

> Yes, I think we should just do one KIP for all the SMTs. You don't 
> have to implement everything all at once or by yourself, but I don't 
> see why we should require one or more follow-up KIPs to apply the 
> exact same changes to the SMTs we missed the first time.

However, the KIP still omits the MaskField and ValueToKey transformations.
This looks like just a typo to me, should we update the KIP before closing the 
vote?

Thanks,
Greg

On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
wrote:

> Hi Greg,
>
> The same check was made by Chris Egerton during the discussion thread.
>
> The original KIP scope was just for the InsertField, ExtractField 
> SMTs, then we decided to enlarge the scope of only the KIP to also 
> other potential affected SMTs.
>
> As of now the PR scope, instead, is only for InsertField and 
> ExtractField SMTs.
>
> Thanks,
> Mario.
>
>
> On Wed, Jun 12, 2024 at 12:08 AM Greg Harris 
>  >
> wrote:
>
> > Hi Mario,
> >
> > Thanks for the KIP. I'm on-board with this KIP, I just wanted to 
> > verify a discrepancy I noticed.
> >
> > I checked all of the call-sites of Struct#get(Field) and
> Struct#get(String)
> > in Kafka, and noticed there are some call-sites which are not 
> > included in the KIP.
> > 1. The Flatten transformation seems to already have the 
> > "replace.null.with.default=false" behavior unconditionally.
> > 2. The MaskField transformation unconditionally injects default 
> > values
> for
> > top-level structs.
> > 3. The ValueTokey transformation injects defaults for each of the 
> > configured "fields"
> > 4. The Values#convertToString method injects default values, but as 
> > this isn't configurable I think we'll need to leave it as-is.
> >
> > Did you intend to add this configuration to the Flatten, MaskField, 
> > or ValueToKey transformations, or should those be addressed in a 
> > separate effort?
> > Since the "false" behavior is desirable, I think the Flatten 
> > transform could be left as-is, rather than adding a configuration 
> > that nobody
> needs.
> >
> > Thanks,
> > Greg
> >
> > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya  wrote:
> >
> > > Thanks for the KIP!
> > >
> > > +1 (binding)
> > >
> > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale 
> > >  >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Just wanted to bump up this thread for visibility.
> > > >
> > > > Thanks!
> > > >
> > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > mickael.mai...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hi Mario,
> > > > >
> > > > > +1 (binding)
> > > > > Thanks for the KIP!
> > > > >
> > > > > Mickael
> > > > >
> > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > mvit...@redhat.com>
> > > > > wrote:
> > > > > >
> > > > > > After 7 days I received only one vote. Should I suppose this 
> > > > > > will
> > not
> > > > be
> > > > > > approved?
> > > > > >
> > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> > >  > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks for the KIP! +1 (binding)
> > > > > > >
> > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > > mvit...@redhat.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > I'd like to call a vote on KIP-1040 which aims to 
> > > > > > > > improve
> > > handling
> > > > of
> > > > > > > > nullable values in InsertField, ExtractField, and other
> > > > > transformations
> > > > > > > >
> > > > > > > > KIP -
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> i.apache.org%2Fconfluence%2Fpages%2Fviewpage.action%3FpageId%3D3037946
> 77=05%7C02%7Cpunsak%40mfec.co.th%7C36a25ef5a026433cbcc808dc8b0323
> c8%7C74105ed972ff4685915475f7408b6f67%7C1%7C0%7C638538092832326790%7CU
> nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> WwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=AWg1i6RQZdMsijQ0Ew0Tr3sj7bDYUsfi%
> 2B%2BVxyg%2BT2wo%3D=0
> > > > > > > >
> > > > > > > > Discussion thread -
> > > > > > > >
> > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli
> > sts.apache.org%2Fthread%2Fggqqqjbg6ccpz8g6ztyj7oxr80q5184n=05%7
> >

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-12 Thread Greg Harris 
Hi Mario,

Thank you for mentioning the earlier DISCUSS thread. I found this comment
from Chris, which was agreed upon and applied to the KIP:

> Yes, I think we should just do one KIP for all the SMTs. You don't have to
> implement everything all at once or by yourself, but I don't see why we
> should require one or more follow-up KIPs to apply the exact same changes
> to the SMTs we missed the first time.

However, the KIP still omits the MaskField and ValueToKey transformations.
This looks like just a typo to me, should we update the KIP before closing
the vote?

Thanks,
Greg

On Wed, Jun 12, 2024 at 6:50 AM Mario Fiore Vitale 
wrote:

> Hi Greg,
>
> The same check was made by Chris Egerton during the discussion thread.
>
> The original KIP scope was just for the InsertField, ExtractField SMTs,
> then we decided to enlarge the scope of only the KIP to also other
> potential affected SMTs.
>
> As of now the PR scope, instead, is only for InsertField and ExtractField
> SMTs.
>
> Thanks,
> Mario.
>
>
> On Wed, Jun 12, 2024 at 12:08 AM Greg Harris  >
> wrote:
>
> > Hi Mario,
> >
> > Thanks for the KIP. I'm on-board with this KIP, I just wanted to verify a
> > discrepancy I noticed.
> >
> > I checked all of the call-sites of Struct#get(Field) and
> Struct#get(String)
> > in Kafka, and noticed there are some call-sites which are not included in
> > the KIP.
> > 1. The Flatten transformation seems to already have the
> > "replace.null.with.default=false" behavior unconditionally.
> > 2. The MaskField transformation unconditionally injects default values
> for
> > top-level structs.
> > 3. The ValueTokey transformation injects defaults for each of the
> > configured "fields"
> > 4. The Values#convertToString method injects default values, but as this
> > isn't configurable I think we'll need to leave it as-is.
> >
> > Did you intend to add this configuration to the Flatten, MaskField, or
> > ValueToKey transformations, or should those be addressed in a separate
> > effort?
> > Since the "false" behavior is desirable, I think the Flatten transform
> > could be left as-is, rather than adding a configuration that nobody
> needs.
> >
> > Thanks,
> > Greg
> >
> > On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya  wrote:
> >
> > > Thanks for the KIP!
> > >
> > > +1 (binding)
> > >
> > > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale  >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Just wanted to bump up this thread for visibility.
> > > >
> > > > Thanks!
> > > >
> > > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> > mickael.mai...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hi Mario,
> > > > >
> > > > > +1 (binding)
> > > > > Thanks for the KIP!
> > > > >
> > > > > Mickael
> > > > >
> > > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > > mvit...@redhat.com>
> > > > > wrote:
> > > > > >
> > > > > > After 7 days I received only one vote. Should I suppose this will
> > not
> > > > be
> > > > > > approved?
> > > > > >
> > > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> > >  > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks for the KIP! +1 (binding)
> > > > > > >
> > > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > > mvit...@redhat.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > I'd like to call a vote on KIP-1040 which aims to improve
> > > handling
> > > > of
> > > > > > > > nullable values in InsertField, ExtractField, and other
> > > > > transformations
> > > > > > > >
> > > > > > > > KIP -
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > > > >
> > > > > > > > Discussion thread -
> > > > > > > >
> > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > > > >
> > > > > > > > Thanks and regards,
> > > > > > > > Mario
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Mario Fiore Vitale
> > > > > >
> > > > > > Senior Software Engineer
> > > > > >
> > > > > > Red Hat 
> > > > > > 
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Mario Fiore Vitale
> > > >
> > > > Senior Software Engineer
> > > >
> > > > Red Hat 
> > > > 
> > > >
> > >
> >
>
>
> --
>
> Mario Fiore Vitale
>
> Senior Software Engineer
>
> Red Hat 
> 
>

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-12 Thread Mario Fiore Vitale 
Hi Greg,

The same check was made by Chris Egerton during the discussion thread.

The original KIP scope was just for the InsertField, ExtractField SMTs,
then we decided to enlarge the scope of only the KIP to also other
potential affected SMTs.

As of now the PR scope, instead, is only for InsertField and ExtractField
SMTs.

Thanks,
Mario.


On Wed, Jun 12, 2024 at 12:08 AM Greg Harris 
wrote:

> Hi Mario,
>
> Thanks for the KIP. I'm on-board with this KIP, I just wanted to verify a
> discrepancy I noticed.
>
> I checked all of the call-sites of Struct#get(Field) and Struct#get(String)
> in Kafka, and noticed there are some call-sites which are not included in
> the KIP.
> 1. The Flatten transformation seems to already have the
> "replace.null.with.default=false" behavior unconditionally.
> 2. The MaskField transformation unconditionally injects default values for
> top-level structs.
> 3. The ValueTokey transformation injects defaults for each of the
> configured "fields"
> 4. The Values#convertToString method injects default values, but as this
> isn't configurable I think we'll need to leave it as-is.
>
> Did you intend to add this configuration to the Flatten, MaskField, or
> ValueToKey transformations, or should those be addressed in a separate
> effort?
> Since the "false" behavior is desirable, I think the Flatten transform
> could be left as-is, rather than adding a configuration that nobody needs.
>
> Thanks,
> Greg
>
> On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya  wrote:
>
> > Thanks for the KIP!
> >
> > +1 (binding)
> >
> > On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale 
> > wrote:
> >
> > > Hi all,
> > >
> > > Just wanted to bump up this thread for visibility.
> > >
> > > Thanks!
> > >
> > > On Wed, May 29, 2024 at 5:31 PM Mickael Maison <
> mickael.mai...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi Mario,
> > > >
> > > > +1 (binding)
> > > > Thanks for the KIP!
> > > >
> > > > Mickael
> > > >
> > > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> > mvit...@redhat.com>
> > > > wrote:
> > > > >
> > > > > After 7 days I received only one vote. Should I suppose this will
> not
> > > be
> > > > > approved?
> > > > >
> > > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
> >  > > >
> > > > > wrote:
> > > > >
> > > > > > Thanks for the KIP! +1 (binding)
> > > > > >
> > > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > > mvit...@redhat.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi everyone,
> > > > > > >
> > > > > > > I'd like to call a vote on KIP-1040 which aims to improve
> > handling
> > > of
> > > > > > > nullable values in InsertField, ExtractField, and other
> > > > transformations
> > > > > > >
> > > > > > > KIP -
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > > >
> > > > > > > Discussion thread -
> > > > > > >
> https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > > >
> > > > > > > Thanks and regards,
> > > > > > > Mario
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Mario Fiore Vitale
> > > > >
> > > > > Senior Software Engineer
> > > > >
> > > > > Red Hat 
> > > > > 
> > > >
> > > >
> > >
> > > --
> > >
> > > Mario Fiore Vitale
> > >
> > > Senior Software Engineer
> > >
> > > Red Hat 
> > > 
> > >
> >
>


-- 

Mario Fiore Vitale

Senior Software Engineer

Red Hat

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-11 Thread Greg Harris 
Hi Mario,

Thanks for the KIP. I'm on-board with this KIP, I just wanted to verify a
discrepancy I noticed.

I checked all of the call-sites of Struct#get(Field) and Struct#get(String)
in Kafka, and noticed there are some call-sites which are not included in
the KIP.
1. The Flatten transformation seems to already have the
"replace.null.with.default=false" behavior unconditionally.
2. The MaskField transformation unconditionally injects default values for
top-level structs.
3. The ValueTokey transformation injects defaults for each of the
configured "fields"
4. The Values#convertToString method injects default values, but as this
isn't configurable I think we'll need to leave it as-is.

Did you intend to add this configuration to the Flatten, MaskField, or
ValueToKey transformations, or should those be addressed in a separate
effort?
Since the "false" behavior is desirable, I think the Flatten transform
could be left as-is, rather than adding a configuration that nobody needs.

Thanks,
Greg

On Tue, Jun 11, 2024 at 9:24 AM Yash Mayya  wrote:

> Thanks for the KIP!
>
> +1 (binding)
>
> On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale 
> wrote:
>
> > Hi all,
> >
> > Just wanted to bump up this thread for visibility.
> >
> > Thanks!
> >
> > On Wed, May 29, 2024 at 5:31 PM Mickael Maison  >
> > wrote:
> >
> > > Hi Mario,
> > >
> > > +1 (binding)
> > > Thanks for the KIP!
> > >
> > > Mickael
> > >
> > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale <
> mvit...@redhat.com>
> > > wrote:
> > > >
> > > > After 7 days I received only one vote. Should I suppose this will not
> > be
> > > > approved?
> > > >
> > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton
>  > >
> > > > wrote:
> > > >
> > > > > Thanks for the KIP! +1 (binding)
> > > > >
> > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > mvit...@redhat.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I'd like to call a vote on KIP-1040 which aims to improve
> handling
> > of
> > > > > > nullable values in InsertField, ExtractField, and other
> > > transformations
> > > > > >
> > > > > > KIP -
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > >
> > > > > > Discussion thread -
> > > > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > >
> > > > > > Thanks and regards,
> > > > > > Mario
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Mario Fiore Vitale
> > > >
> > > > Senior Software Engineer
> > > >
> > > > Red Hat 
> > > > 
> > >
> > >
> >
> > --
> >
> > Mario Fiore Vitale
> >
> > Senior Software Engineer
> >
> > Red Hat 
> > 
> >
>

RE: Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-11 Thread Punsak Incham 
Thanks for the KIP!
+1 (Non-binding)


On 2024/06/11 16:23:38 Yash Mayya wrote:
> Thanks for the KIP!
>
> +1 (binding)
>
> On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale 
> mv...@redhat.com
> wrote:
>
> > Hi all,
> >
> > Just wanted to bump up this thread for visibility.
> >
> > Thanks!
> >
> > On Wed, May 29, 2024 at 5:31 PM Mickael Maison 
> > mi...@gmail.com
> > wrote:
> >
> > > Hi Mario,
> > >
> > > +1 (binding)
> > > Thanks for the KIP!
> > >
> > > Mickael
> > >
> > > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale 
> > > mv...@redhat.com
> > > wrote:
> > > >
> > > > After 7 days I received only one vote. Should I suppose this will not
> > be
> > > > approved?
> > > >
> > > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton  > >
> > > > wrote:
> > > >
> > > > > Thanks for the KIP! +1 (binding)
> > > > >
> > > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> > mvit...@redhat.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I'd like to call a vote on KIP-1040 which aims to improve handling
> > of
> > > > > > nullable values in InsertField, ExtractField, and other
> > > transformations
> > > > > >
> > > > > > KIP -
> > > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > > >
> > > > > > Discussion thread -
> > > > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > > >
> > > > > > Thanks and regards,
> > > > > > Mario
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Mario Fiore Vitale
> > > >
> > > > Senior Software Engineer
> > > >
> > > > Red Hat https://www.redhat.com/
> > > > https://www.redhat.com/
> > >
> > >
> >
> > --
> >
> > Mario Fiore Vitale
> >
> > Senior Software Engineer
> >
> > Red Hat https://www.redhat.com/
> > https://www.redhat.com/
> >
>



Punsak Incham
Data Platform Engineer (Kafka and Confluent)
MFEC

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-11 Thread Yash Mayya 
Thanks for the KIP!

+1 (binding)

On Tue, Jun 11, 2024 at 7:40 PM Mario Fiore Vitale 
wrote:

> Hi all,
>
> Just wanted to bump up this thread for visibility.
>
> Thanks!
>
> On Wed, May 29, 2024 at 5:31 PM Mickael Maison 
> wrote:
>
> > Hi Mario,
> >
> > +1 (binding)
> > Thanks for the KIP!
> >
> > Mickael
> >
> > On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale 
> > wrote:
> > >
> > > After 7 days I received only one vote. Should I suppose this will not
> be
> > > approved?
> > >
> > > On Mon, May 20, 2024 at 4:14 PM Chris Egerton  >
> > > wrote:
> > >
> > > > Thanks for the KIP! +1 (binding)
> > > >
> > > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale <
> mvit...@redhat.com
> > >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I'd like to call a vote on KIP-1040 which aims to improve handling
> of
> > > > > nullable values in InsertField, ExtractField, and other
> > transformations
> > > > >
> > > > > KIP -
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > > >
> > > > > Discussion thread -
> > > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > > >
> > > > > Thanks and regards,
> > > > > Mario
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Mario Fiore Vitale
> > >
> > > Senior Software Engineer
> > >
> > > Red Hat 
> > > 
> >
> >
>
> --
>
> Mario Fiore Vitale
>
> Senior Software Engineer
>
> Red Hat 
> 
>

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-06-11 Thread Mario Fiore Vitale 
Hi all,

Just wanted to bump up this thread for visibility.

Thanks!

On Wed, May 29, 2024 at 5:31 PM Mickael Maison 
wrote:

> Hi Mario,
>
> +1 (binding)
> Thanks for the KIP!
>
> Mickael
>
> On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale 
> wrote:
> >
> > After 7 days I received only one vote. Should I suppose this will not be
> > approved?
> >
> > On Mon, May 20, 2024 at 4:14 PM Chris Egerton 
> > wrote:
> >
> > > Thanks for the KIP! +1 (binding)
> > >
> > > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale  >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I'd like to call a vote on KIP-1040 which aims to improve handling of
> > > > nullable values in InsertField, ExtractField, and other
> transformations
> > > >
> > > > KIP -
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > > >
> > > > Discussion thread -
> > > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > > >
> > > > Thanks and regards,
> > > > Mario
> > > >
> > >
> >
> >
> > --
> >
> > Mario Fiore Vitale
> >
> > Senior Software Engineer
> >
> > Red Hat 
> > 
>
>

-- 

Mario Fiore Vitale

Senior Software Engineer

Red Hat

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-05-29 Thread Mickael Maison 
Hi Mario,

+1 (binding)
Thanks for the KIP!

Mickael

On Mon, May 27, 2024 at 12:06 PM Mario Fiore Vitale  wrote:
>
> After 7 days I received only one vote. Should I suppose this will not be
> approved?
>
> On Mon, May 20, 2024 at 4:14 PM Chris Egerton 
> wrote:
>
> > Thanks for the KIP! +1 (binding)
> >
> > On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale 
> > wrote:
> >
> > > Hi everyone,
> > >
> > > I'd like to call a vote on KIP-1040 which aims to improve handling of
> > > nullable values in InsertField, ExtractField, and other transformations
> > >
> > > KIP -
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> > >
> > > Discussion thread -
> > > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> > >
> > > Thanks and regards,
> > > Mario
> > >
> >
>
>
> --
>
> Mario Fiore Vitale
>
> Senior Software Engineer
>
> Red Hat 
>

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-05-27 Thread Mario Fiore Vitale 
After 7 days I received only one vote. Should I suppose this will not be
approved?

On Mon, May 20, 2024 at 4:14 PM Chris Egerton 
wrote:

> Thanks for the KIP! +1 (binding)
>
> On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale 
> wrote:
>
> > Hi everyone,
> >
> > I'd like to call a vote on KIP-1040 which aims to improve handling of
> > nullable values in InsertField, ExtractField, and other transformations
> >
> > KIP -
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
> >
> > Discussion thread -
> > https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
> >
> > Thanks and regards,
> > Mario
> >
>


-- 

Mario Fiore Vitale

Senior Software Engineer

Red Hat

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

2024-05-20 Thread Chris Egerton 
Thanks for the KIP! +1 (binding)

On Mon, May 20, 2024 at 4:22 AM Mario Fiore Vitale 
wrote:

> Hi everyone,
>
> I'd like to call a vote on KIP-1040 which aims to improve handling of
> nullable values in InsertField, ExtractField, and other transformations
>
> KIP -
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677
>
> Discussion thread -
> https://lists.apache.org/thread/ggqqqjbg6ccpz8g6ztyj7oxr80q5184n
>
> Thanks and regards,
> Mario
>