Re: Do we want to add more SMTs to Apache Kafka?

2021-12-29 Thread Chris Egerton
Hi all, I think restricting the set of out-of-the-box SMTs that we provide with Connect is reasonable. I do think Joshua raises a valuable point, though. At the risk of reiterating his ideas, we can gain a few things from improving the existing SMTs provided with Connect: first, we can establish

Re: Do we want to add more SMTs to Apache Kafka?

2021-11-21 Thread Joshua Grisham
Hi all, >From my perspective I think that the type of transformations which are already covered by the existing SMTs is quite good (but anyone else please say if you feel like you are missing something that feels "standard"), but the biggest issue is the limitations that many of them have which

Re: Do we want to add more SMTs to Apache Kafka?

2021-11-19 Thread Brandon Brown
I agree, if the desire is to keep the internal SMTs collection small then providing an ease of discovery like Gunnar suggestions would be extremely helpful. Brandon Brown > On Nov 19, 2021, at 6:13 PM, Gunnar Morling > wrote: > > Hi all, > > Just came across this thread, I hope the late

Re: Do we want to add more SMTs to Apache Kafka?

2021-11-19 Thread Gunnar Morling
Hi all, Just came across this thread, I hope the late reply is ok. FWIW, we're in a similar situation in Debezium, where users often request new (Debezium-specific) SMTs, and we generally tend to recommend them to be maintained by users themselves, unless they are truly generic. This excludes a

Re: Do we want to add more SMTs to Apache Kafka?

2021-11-07 Thread Brandon Brown
I like the idea of a select number of SMTs being offered and supported out of the box. The addition of SMTs via this process is nice because it allows for a rich set to be supported out of the box and without the need for extra work to deploy. Perhaps this is a spot where the community could

Do we want to add more SMTs to Apache Kafka?

2021-11-07 Thread Randall Hauch
We have had several requests to add more Connect Single Message Transforms (SMTs) to the project. When SMTs were first introduced with KIP-66 (ref 1) in Jun 2017, the KIP mentioned the following: > Criteria: SMTs that are shipped with Kafka Connect should be general enough > to apply to many