Hi all, Because the Spark Streaming direct Kafka consumer maps offsets for a given Kafka topic and a partition internally while having enable.auto.commit set to false, how can I retrieve the offset of each made consumer’s poll call using the offset ranges of an RDD? More precisely, the information I seek to get after each poll call is the following: <timestamp, offset, partition>.
Thanks in advance, Dominik