Hi, I have bene trying to use kafka-producer-perf-test.sh to arrive at certain benchmarks. When I try to run it with --compression-codec values of 1, 2 and 3, I notice increased throughput compared to NoCompressionCodec
But, When I checked the Producerperformance.scala, I saw that the the `producer.send` is getting data from the method: `generateProducerData`. But, this data is just an empty array of Bytes. Now, as per my basic understanding of compression algorithms, I think a byte sequence of zeros will eventually result in a very small message, because of which I thought I might be observing better throughput. So, in line: 247 of ProducerPerformance.scala, I did this minor code change:- *val message = "qopwr11591UPD113582260001AS1IL1-1N/A1Entertainment1-1an-example.com1-1-1-1-1-1-1-1011413/011413_factor_points_FNC_,LOW,MED_LOW,MED,HIGH,HD,.mp4.csmil/bitrate=11subcategory 71Title 10^D1-1-111-1-1-1-1-1-111-1-1-1-1-115101-1-1-1-1126112491-1-1-1-1-1-1-1-1-1-1-1-1-1-1-111-1-1-r1VR-11591UPD113582260001AS1IL1-1N/A1Entertainment1-1an-example.com1-1-1-1-1-1-1-1011413/011413_factor_points_FNC_,LOW,MED_LOW,MED,HIGH,HD,.mp4.csmil/bitrate=11subcategory 71Title 10^D1-1-111-1-1-1-1-1-111-1-1-1-1-115101-1-1-1-1126112491-1-1-1-1-1-1-1-1-1-1-1-1-1-1-111-1-1-r1VR-11591UPD113582260001AS1IL1-1N/A1Entertainment1-1an-example.com1-1-1-1-1-1-1-1011413/011413_factor_points_FNC_,LOW,MED_LOW,MED,HIGH,HD,.mp4.csmil/bitrate=11subcategory 71Title 10^D1-1-111-1-1-1-1-1-111-1-1-1-1-115101-1-1-1-1126112491-1-1-1-1-1-1-1-1-1-1-1-1-1-1-111-1-1-"message.getBytes().slice(0,msgSize)* This makes sure that I have a big message, and I can slice that message to the message size passed in the command line options But, the problem is that when I try running the same with --compression-codec vlues of 1, 2 or 3, I still am seeing ASCII data (i.e. uncompressed one only) I want to ask whether this is a bug. And, using kafka-producer-perf-test.sh, how can I send my own compressed data ? Thanks, Prabhjot
