Does Kafka dependency jars changed for Spark Structured Streaming 2.2.0?

[kant kodali]

Hi All,

Does Kafka dependency jars changed for Spark Structured Streaming 2.2.0?

kafka-clients-0.10.0.1.jar

spark-streaming-kafka-0-10_2.11-2.2.0.jar


1) Above two are the only Kafka related jars or am I missing something?

2)  What is the difference between the above two jars?

3) If I have the following code. Which jar do I use? It looks like I was
able to compile if any one of the jars above thats why its a bit confusing.


StreamingQuery query = sparkSession.readStream()
        .format("kafka")
        .option("kafka.bootstrap.servers", "localhost:9092")
        .option("subscribe", "hello"))
        .option("startingOffsets", "earliest")
        .load()
        .writeStream()
        .format("console")
        .start();

query.awaitTermination();


Thanks