Hi All, I have read this posting from linkedIn Team member; http://geekmantra.wordpress.com/2013/03/28/compression-in-kafka-gzip-or-snappy/ ; Thanks.
I have few questions and thoughts: 1) In asynchronous mode, you are compressing the batch, not the individual send. 2) There is no way we can ask for the compressed message from the consumer end — because that is want we needed. But I am not sure though that is going to work since we we may not get the messages in the exact same sizes that they are compressed. 3) Since you are doing another round of uncompress/compress in the broker to help maintain the message offsets, will the larger individual messages help reduce the CUP load on the Broker. Because that would effectively reduces the number of offsets and therefore individual uncompress/compress activities. If so we can batch up even before calling the send to make the message sizes become larger. 4) Has any one else done Snappy Vs. GZIP comparison and chosen one over the other. If so what are your findings and why if you could share as we are going through this expertise. Thanks for your thoughts, Maung
