Thank you Stig for that explanation. How does the new Consumer handle failure of Kafka brokers? If I connect to one broker on 9092 and that brokers dies, how will it know to connect to the other on 9093? Is it because it knows that consumer group belongs to two brokers?
Thanks Pavel On Sun, May 20, 2018 at 10:55 AM Stig Rohde Døssing <s...@apache.org> wrote: > The storm-kafka-client module is intended as the replacement for the > storm-kafka module. The storm-kafka module uses a Kafka client that is now > deprecated ( > https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/SimpleConsumer.scala) > under the hood. Storm-kafka-client uses > https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html > instead. > > If you're writing new code, you should use storm-kafka-client. We're > planning to remove storm-kafka in Storm 2.0.0, since the underlying client > has been deprecated and will probably be removed by Kafka at some point. > > 2018-05-20 16:37 GMT+02:00 Pavel Sapozhnikov <pavel.sapozhni...@gmail.com> > : > >> I am sure this has probably been answered before somewhere but I can't >> find a correct answer. >> >> What is the difference between >> >> *org.apache.storm.kafka.spout.KafkaSpout* >> >> and >> >> *org.apache.storm.kafka.KafkaSpout* >> >> In my experience the first one uses *KafkaSpoutConfig *to connect to >> broker directly while the second one connects to Zookeeper and resolves to >> a leader broker. >> >> Correct me if I am wrong. Is one of the old way of doing things versus >> new way? >> >> If I have very simplistic two brokers connected to one zk and a topic say >> X replicated between the two, in the second way, I only have to connect to >> ZK but in the first way I have to connect to broker directly. >> >> What if I have two brokers as I mentioned above do I need two storm >> spouts then for the case of storm.kafka.KafkaSpout? If that's the case >> would I need to set something up to *not *process duplicate messages >> from two spouts? >> >> Again if I got something wrong or misunderstanding something please >> correct me if I am wrong. >> > >
