Hi I hope I am using the right mailing list. Please advice if I am wrong.
I have few observations about the KafkaSpout and feel that some of these lead to inefficiencies. It will be of great help if someone can throw some light on the rationale behind the implementation. 1) Kafka polling and committing offsets is done in the spout thread which is somewhat against the spout best practices. Is simplicity the reason behind this design? Am I missing something? 2) Poll-iterate-commit-seek loop seems inefficient in recurrent failure scenarios. Let say the first massage fails. We will keep polling the same set of messages at least as many times as that message is retried and probably more if we are using exponential back-off. Did I misunderstand the implementation? Regards Chandan
