I'm using Kafka direct stream (auto.offset.reset = earliest) and enable
Spark streaming's checkpoint.
The application starts and consumes messages correctly. Then I stop the
application and clean the checkpoint folder.
I restart the application and expect it to consumes old messages. But it
doesn't consume any data. And there are logs as below:
[org.apache.spark.streaming.kafka010.KafkaRDD] (Executor task
launch worker-0;) Beginning offset 25 is the same as ending offset
skipping aa 0
So I think the offset is stored not only in checkpoint but also in
Kafka, right?
Is it because I'm using the same group.id? How can I delete the consumer
group manually?
Thanks again for any help!
