Hi Everyone, I am attempting to use the KafkaBolt for output from my Storm topology. Whereas the throughput from the KafkaSpout is screaming fast, output to Kafka via the KafkaBolt is comparatively really slow. I've tried outputting with request.required.acks set to 0,1,-1 which does not affect throughput significantly.
I am thinking that one issue is that the KafkaSpout is reading off of a topic with 20 partitions, whereas the KafkaBolt is writing to a topic with 10 partitions. I am going to expand the write topic to twenty partitions tomorrow and retry as I think that increasing the producer group size via a greater number of partitions should help. Any other ideas/guidance would be great--thanks! --John
