Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2021-12-07 Thread Mickael Maison
Thanks Joshua for this KIP, it looks like a useful addition. I have a few questions: 1. In Public Interfaces and Proposed Changes, the KIP mentions the current "field" setting will be renamed to "fields". Do you instead mean a new setting "fields" will be added and the existing setting will be

Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2021-11-22 Thread Joshua Grisham
Hi Randall, thanks again for the great feedback! Sorry it took me a minute to get back on this one--I have been a bit full up over the last few months--but now I will try to brush it off and see if we can figure out if it makes sense to move forward and possibly close this one out. Regarding

Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2021-07-28 Thread Randall Hauch
Thanks for the contribution, Joshua. Overall I think this is a really good improvement to this connector, for all the reasons cited in the motivation section. I do have some requests, though. The biggest issue I have is that the change is not backward compatible. We've worked really hard to make

Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2021-05-15 Thread Joshua Grisham
Hello again Kafka Developers community! I thought it has been some time and I would try to dust KIP-682 off and see if anyone wanted to take a discussion on them, if it still makes sense or if this should go in a different direction. Anyone have thoughts on this? That Connect's

Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2020-11-07 Thread Joshua Grisham
Hi Brandon, I have added what i called "recursive" support to find child fields on any level to the Cast and ReplaceField transforms as well, the PR is here https://github.com/apache/kafka/pull/9493 I plan to try and write up a KIP for that one as well maybe in the next day or two. It is not

Re: [DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2020-11-07 Thread Brandon Brown
I love this idea! I have a current KIP for a hash transform and am working adding multi field/nested support to that one. I can think of a few times we’ve needed functionality like that. I’d add that adding support for transforming nested fields would be a great feature for this. -Brandon

[DISCUSS] KIP-682: Connect TimestampConverter support for multiple fields and multiple input formats

2020-11-07 Thread Joshua Grisham
Hello everyone! (Thanks to Tom Bentley for directing me in this direction!) I have made a series of changes to some of the standard Connect transforms to meet some of the challenges at my company to consume messages using Connect, and have been running them for a few weeks now as custom SMTs. I