Re: Kafka streaming receiver approach - new topic not read from beginning

I successfully processed my data by resetting manually my topic offsets on ZK. If it may help someone, here's my steps : Make sure you stop all your consumers before doing that, otherwise they overwrite the new offsets you wrote set

Re: Kafka streaming receiver approach - new topic not read from beginning

Thanks for your quick answer. If I set "auto.offset.reset" to "smallest" as for KafkaParams like this val kafkaParams = Map[String, String]( "metadata.broker.list" -> brokers, "group.id" -> groupId, "auto.offset.reset" -> "smallest" ) And then use : val streams =

Re: Kafka streaming receiver approach - new topic not read from beginning

You could set this configuration "auto.offset.reset" through parameter "kafkaParams" which is provided in some other overloaded APIs of createStream. By default Kafka will pick data from latest offset unless you explicitly set it, this is the behavior Kafka, not Spark. Thanks Saisai On Mon, Feb

Kafka streaming receiver approach - new topic not read from beginning

Hi, Do you know why, with the receiver approach and a *consumer group*, a new topic is not read from the beginning but from the lastest ? Code example : val kafkaStream =