Hi, Just found out that KafkaSource [1] does not use KafkaSourceInitialOffsetWriter (of KafkaMicroBatchStream) [2] for initial offsets.
Any reason for that? Should I report an issue? Just checking out as I'm with 2.4.3 exclusively and have no idea what's coming for 3.0. [1] https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala#L102 [2] https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchStream.scala#L281 Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski The Internals of Spark SQL https://bit.ly/spark-sql-internals The Internals of Spark Structured Streaming https://bit.ly/spark-structured-streaming The Internals of Apache Kafka https://bit.ly/apache-kafka-internals Follow me at https://twitter.com/jaceklaskowski
