Thanks Liam....essentially, it would be an internal topic that we would be creating to use as a cache store by accessing topic through a GlobalKTable, so the problem you mentioned above for storing Hashmap may not apply there
On Tue, May 12, 2020, 21:25 Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote: > Hi Pushkar, > > Just wanted to say, as someone with battle scars from ActiveMQ and Camel, > there's very many good reasons to avoid Java serialization on a messaging > system. What if you need to tail a topic from the console? What if your > testers want to access in their pytests? Etc. And that's not even getting > into the minefield of Java serialization compatibility between one VM and > another. > > That said, if using another serialization format like Avro or JSON isn't > feasible, you can implement your own serializer/deserializer. > > At it's heart, a Kafka producer sends a byte array and a consumer receives > a byte array. So a custom serialiser to turn a list or map into bytes is > easily doable using an ObjectInputStream. And a consumer can use > deserialiser built on an ObjectOutputStream at the other end. > > Kind regards, > > Liam Clarke-Hutchinson > > On Tue, 12 May 2020, 5:13 pm Pushkar Deole, <pdeole2...@gmail.com> wrote: > > > And by the way, confluent has provided KafkaAvroSerializer/Deserialier. > > Can't they be used to do conversion for java types? > > > > On Tue, May 12, 2020 at 10:09 AM Pushkar Deole <pdeole2...@gmail.com> > > wrote: > > > > > Ok... so jackson json serialization is the way to go for hashmaps as > > well? > > > > > > On Mon, May 11, 2020 at 7:57 PM John Roesler <vvcep...@apache.org> > > wrote: > > > > > >> Oh, my mistake. I thought this was a different thread :) > > >> > > >> You might want to check, but I don’t think there is a kip for a map > > >> serde. Of course, you’re welcome to start one. > > >> > > >> Thanks, > > >> John > > >> > > >> On Mon, May 11, 2020, at 09:14, John Roesler wrote: > > >> > Hi Pushkar, > > >> > > > >> > I don’t think there is. You’re welcome to start one if you think it > > >> > would be a useful addition. > > >> > > > >> > Before worrying about it further, though, you might want to check > the > > >> > InMemoryKeyValueStore implementation, since my answer was from > memory. > > >> > > > >> > Thanks, > > >> > John > > >> > > > >> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote: > > >> > > John, > > >> > > is there KIP in progress for supporting Java HashMap also? > > >> > > > > >> > > On Sun, May 10, 2020, 00:47 John Roesler <vvcep...@apache.org> > > wrote: > > >> > > > > >> > > > Yes, that’s correct. It’s only for serializing the java type > > >> ‘byte[]’. > > >> > > > > > >> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote: > > >> > > > > Thanks John... I got to finish the work in few days so need to > > >> get it > > >> > > > > quick, so looking for something ready. I will take a look at > > >> jackson > > >> > > > json. > > >> > > > > > > >> > > > > By the way, what is the byteArrayserializer? As the name > > >> suggests, it is > > >> > > > > for byte arrays so won't work for java ArrayList, right? > > >> > > > > > > >> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler < > > vvcep...@apache.org> > > >> wrote: > > >> > > > > > > >> > > > > > Hi Pushkar, > > >> > > > > > > > >> > > > > > If you’re not too concerned about compactness, I think > Jackson > > >> json > > >> > > > > > serialization is the easiest way to serialize complex types. > > >> > > > > > > > >> > > > > > There’s also a kip in progress to add a list serde. You > might > > >> take a > > >> > > > look > > >> > > > > > at that proposal for ideas to write your own. > > >> > > > > > > > >> > > > > > Thanks, > > >> > > > > > John > > >> > > > > > > > >> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote: > > >> > > > > > > Won't say it's a good idea to use java serialized classes > > for > > >> > > > messages, > > >> > > > > > but > > >> > > > > > > you should use a byteArraySerializer if you want to do > such > > >> things > > >> > > > > > > > > >> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole < > > >> pdeole2...@gmail.com> a > > >> > > > > > écrit : > > >> > > > > > > > > >> > > > > > > > Hi All, > > >> > > > > > > > > > >> > > > > > > > I have a requirement to store a record with key as java > > >> String and > > >> > > > > > value as > > >> > > > > > > > java's ArrayList in the kafka topic. Kafka has by > default > > >> provided > > >> > > > a > > >> > > > > > > > StringSerializer and StringDeserializer, however for > java > > >> > > > ArrayList, > > >> > > > > > how > > >> > > > > > > > can get serializer. Do I need to write my own? Can > someone > > >> share if > > >> > > > > > someone > > >> > > > > > > > already has written one? > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > -- > > >> > > > > > > *Nicolas Carlot* > > >> > > > > > > > > >> > > > > > > Lead dev > > >> > > > > > > | | nicolas.car...@chronopost.fr > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost > > >> déménage. > > >> > > > La > > >> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 > > Paris* > > >> > > > > > > > > >> > > > > > > [image: Logo Chronopost] > > >> > > > > > > | chronopost.fr <http://www.chronopost.fr/> > > >> > > > > > > Suivez nous sur Facebook < > > >> https://fr-fr.facebook.com/chronopost> et > > >> > > > > > Twitter > > >> > > > > > > <https://twitter.com/chronopost>. > > >> > > > > > > > > >> > > > > > > [image: DPD Group] > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
