Hi Matthias, I think you misunderstood my diagram. I wanted to show the next relationships: - Serde is KeySerde and ValueSerde - KeySerde is KeySerializer and KeyDeserializer - ValueSerde is ValueSerializer and ValueDeserializer
So, `Serde` is is most "powerfull" class. It could do anything. `KeySerde` and `ValueSerde` are not specialized `Serde`, but a component of a `Serde`. Implementation of these relationships through inheritance will look like this (pseudocode): class KeySerializer class KeyDeserializer class ValueSerializer class ValueDeserializer class KeySerde extends KeySerializer with KeyDeserializer class ValueSerde extends ValueSerializer with ValueDeserializer class Serde extends KeySerde with ValueSerde So, everywhere where you need, for example, `KeyDeserializer`, you could use `KeyDeserializer`, `KeySerde` or `Serde`. -Yuriy On Fri, Jun 5, 2020 at 7:10 AM Matthias J. Sax <mj...@apache.org> wrote: > As you say "hierarchy" I read this really as "class hierarchy". For this > case, I think that we need to do it differently. > > I agree to this part: > > KeySerde extends Serde > ValueSerde extends Serde > > However, > > KeySerializer extends KeySerde (etc) > > does not make sense IMHO, because a `KeySerializer` is no a special > KeySerde; it's only a serializer but it's not a deserializer. > > In fact the hierarchy goes the other direction: > > Serde extends Serializerd, Deserializer > > > Atm, a Serde is just a "container" for serializers and desrialzier though. > > > -Matthias > > > > On 6/2/20 10:40 PM, Yuriy Badalyantc wrote: > > Hi. > > > > I'm the author of the KIP-616. I got acquainted with the KIP-513 and I > > think overall it's a good idea. But I think workaround only on the scala > > side is not enough. We could consider moving a bit further and change > serde > > hierarchy on the java side to something like this: > > > > KeySerializer-----↘ > > KeySerde---↘ > > KeyDeserializer---↗ ↓ > > Serde > > ValueSerializer---↘ ↑ > > ValueSerde-↗ > > ValueDeserializer-↗ > > > > I understand that this is a bit too revolutionary, but I think this is > the > > right direction. > > > > Regarding KIP-616 and KIP-513 connection. It's hard to say at the current > > moment how should we implement these kips: together or separately. It > looks > > like there is no consensus on the implementation details for these kips. > > > > If we decided to create a new `KeySerde` and `ValueSerde` on the scala > side > > (current plan without my proposal above), we could use their companions > for > > default instances. In this case, it looks like we don't need to do > KIP-616. > > But what about backward compatibility? What should we do with > > `org.apache.kafka.streams.scala.Serdes`? Deprecate it? > > > > - Yuriy > > > > On Sun, May 31, 2020 at 1:24 AM John Roesler <vvcep...@apache.org> > wrote: > > > >> Hi Mykhailo, > >> > >> Wow, I really dropped the ball here. I have just looked over your KIP > >> again, and now I see how you don’t need to change every dsl method, only > >> Consumed, Materialized, etc. > >> > >> I think this would be a good addition. Yuriy has just proposed KIP-616 > to > >> fix some other problems with the implicit serdes. I’m wondering if these > >> two kips have any joint opportunities we should consider, or if it’s > better > >> to continue to consider them separately. > >> > >> Thanks, > >> John > >> > >> On Wed, Jan 22, 2020, at 16:18, Михаил Ерёменко wrote: > >>> Hi, John! > >>> > >>> Sorry for the late reply. I am not really familiar with this mail list > >>> discussions, so I have not seen your mails. > >>> > >>> Regarding your question: > >>>> I guess what > >>> I'm struggling with is why you actually want to have different key > and > >>> serdes for the same type > >>> > >>> I think good example will be (and it is actually what we do in ours > >>> project) using confluent schema registry in conjunction with kafka > >>> streams. Some models can be used as keys as well as values. When we > >>> define schema registry compatible serde, we have to specify is it for > >>> key or not. We can of course create two serdes for the same model, but > >>> in this case implicit semantic will not work because scala doesn’t know > >>> which implicit to pick. And things become even more complicated in case > >>> if you will try to derive your serdes (we derive serdes in ours > >>> project). > >>> > >>> One more thing: > >>>> every method in the streams-scala DSL. > >>> > >>> So far we've just changed > >>> org.apache.kafka.streams.scala.ImplicitConversions and > >>> org.apache.kafka.streams.scala.kstream.Materialized and it works for > >>> us. Also we did introduce default serdes for primitive types. > >>> > >>> Regards, > >>> Mykhailo > >> > > > >
