E.g., as per
https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html
val df = spark
.read
.format("kafka")
.option("kafka.bootstrap.servers", "host1:port1,host2:port2")
.option("subscribe", "topic1")
.option("startingOffsets", "earliest")
Hello 姜戎 ,
Unfortunately there's not enough information in your email for us to help
you. Are you trying to use Spark Batch to read from Kafka? Have you tried
setting "endingOffsets" to "latest" instead of an arbitrary number?
Kind regards,
Liam Clarke-Hutchinson
On Fri, May 1, 2020 at 2:36
failed to get records for compacted ...after polling for12
partition 0 offset min=0 max=1427265
get offsetrange 0 until 50 to make rdd
