Whatever you use I recommend some sort of wrapper since Kafka doesn't support any sort of metadata (like the version of the serialization format). -Dave
-----Original Message----- From: Matt Farmer [mailto:[email protected]] Sent: Thursday, January 11, 2018 8:56 AM To: [email protected] Subject: Re: Best practice for publishing byte messages to Kafka We use Thrift and Kryo, and we haven’t done any formal analysis recently. Performance numbers for those should be easy to find by googling around. I can say that we push tens of thousands of messages per second at peak and serialization hasn’t been a cause of production lag in the entire year that I’ve been working on this application. That’s an imprecise analysis, but I would readily recommend either library in a case where performance is your only concern. (Though, Avro may prove to be a better choice than Thrift these days.) > On Jan 9, 2018, at 11:17 PM, Ali Nazemian <[email protected]> wrote: > > Thanks, Matt. Have you done any benchmarking to see how using > different Serializers may impact throughput/latency? > > Regards, > Ali > > On Wed, Jan 10, 2018 at 7:55 AM, Matt Farmer <[email protected]> wrote: > >> We use the default byte array serializer provided with Kafka and it >> works great for us. >> >>> On Jan 9, 2018, at 8:12 AM, Ali Nazemian <[email protected]> wrote: >>> >>> Hi All, >>> >>> I was wondering whether there is any best practice/recommendation >>> for publishing byte messages to Kafka. Is there any specific >>> Serializer that >> is >>> recommended for this matter? >>> >>> Cheers, >>> Ali >> >> > > > -- > A.Nazemian This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
