Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-13 Thread Mario Fiore Vitale
Hi all, Any other suggestions/objections/dubs? On Fri, May 10, 2024 at 5:10 PM Chris Egerton wrote: >  Done > > On Fri, May 10, 2024, 10:55 Mario Fiore Vitale wrote: > > > Thanks a lot! I have just a minor comment, should we also update the > title > > to be more generic since now it's also

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Chris Egerton
 Done On Fri, May 10, 2024, 10:55 Mario Fiore Vitale wrote: > Thanks a lot! I have just a minor comment, should we also update the title > to be more generic since now it's also related to other SMTs? > > On Fri, May 10, 2024 at 4:44 PM Chris Egerton > wrote: > > > I've finished updating the

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Mario Fiore Vitale
Thanks a lot! I have just a minor comment, should we also update the title to be more generic since now it's also related to other SMTs? On Fri, May 10, 2024 at 4:44 PM Chris Egerton wrote: > I've finished updating the KIP; @Mario, please let me know what you think. > > On Fri, May 10, 2024 at

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Chris Egerton
I've finished updating the KIP; @Mario, please let me know what you think. On Fri, May 10, 2024 at 10:26 AM Chris Egerton wrote: > I can do it :) > > On Fri, May 10, 2024 at 10:02 AM Mario Fiore Vitale > wrote: > >> Yes, I agree. Unfortunately due to the issue of the creation of a new >>

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Chris Egerton
I can do it :) On Fri, May 10, 2024 at 10:02 AM Mario Fiore Vitale wrote: > Yes, I agree. Unfortunately due to the issue of the creation of a new > account for the WIKI, I asked Mickael Maison to create the KIP for me. > > I'll ask him to update the KIP. Do you have other alternatives? > >

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Mario Fiore Vitale
Yes, I agree. Unfortunately due to the issue of the creation of a new account for the WIKI, I asked Mickael Maison to create the KIP for me. I'll ask him to update the KIP. Do you have other alternatives? Thanks, Mario. On Fri, May 10, 2024 at 3:40 PM Chris Egerton wrote: > Yes, I think we

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Chris Egerton
Yes, I think we should just do one KIP for all the SMTs. You don't have to implement everything all at once or by yourself, but I don't see why we should require one or more follow-up KIPs to apply the exact same changes to the SMTs we missed the first time. On Fri, May 10, 2024 at 5:20 AM Mario

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-10 Thread Mario Fiore Vitale
Hi Chris, Thanks for the survey. Do you think I need to update the KIP to put all of these? On Thu, May 9, 2024 at 4:14 PM Chris Egerton wrote: > After doing a brief survey of the SMTs that ship with Connect, it seems > like these would also benefit: > > - HeaderFrom, which populates record

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-09 Thread Chris Egerton
After doing a brief survey of the SMTs that ship with Connect, it seems like these would also benefit: - HeaderFrom, which populates record headers with subfields of keys/values [1] - Cast, which performs type transformation on subfields of keys/values [2] - SetSchemaMetadata, which (when the

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-09 Thread Mario Fiore Vitale
Hi Chris, > Wouldn't ValueToKey [1] be applicable as well, for example? Yes, also that one can be affected. On Wed, May 8, 2024 at 5:59 PM Chris Egerton wrote: > Wait, just one more thing--are there any other SMTs that could benefit from > this? Wouldn't ValueToKey [1] be applicable as well,

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-08 Thread Chris Egerton
Wait, just one more thing--are there any other SMTs that could benefit from this? Wouldn't ValueToKey [1] be applicable as well, for example? [1] -

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-08 Thread Chris Egerton
Hi Mario, I think we could have something like `copy` and `copyWithoutDefaults` to get around that, but now that you bring up compatibility, I think it's best to hold off on this. I'm forced to recall that anything we add to the Connect API may be used by plugin developers who write for the

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-08 Thread Mario Fiore Vitale
Hi Chris, Thanks for reviewing this. > It seems like the pattern of "copy the contents of this Struct into another one for the purpose of mutation" could be fairly common in user code bases in addition to the core Connect SMTs. Do you think there's a way to simplify this with, e.g., a

Re: KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-08 Thread Chris Egerton
Hi Mario, Thanks for the KIP! Looks good overall. One quick thought--it wasn't immediately obvious to me why we had to touch on InsertField for this. It looks like the reason is that we use Struct::get [1] to create a clone of the input value [2], instead of Struct::getWithoutDefault [3]. It

KIP-1040: Improve handling of nullable values in InsertField/ExtractField transformations

2024-05-08 Thread Mario Fiore Vitale
Hi All, I have created (through Mickael Maison's account since there was an issue creating a new one for me) KIP-1040[1] to improve handling of nullable values in InsertField/ExtractField transformations, this is required after the introduction of KIP-581[2] that introduced the configuration