What supports Python in (Kafka?) 0.8? I don't think Spark ever had a
specific Python-Kafka integration. But you have always been able to
use it to read DataFrames as in Structured Streaming.
Kafka 0.8 support is deprecated (gone in 3.0) but 0.10 means 0.10+ -
works with the latest 2.x.
What is the
Hey,
Maybe I'm missing some restriction with EMR, but have you tried to use
Structured Streaming instead of Spark Streaming?
https://spark.apache.org/docs/2.4.5/structured-streaming-kafka-integration.html
Regards
On Wed, 12 Aug 2020 at 14:12, Hamish Whittal
wrote:
> Hi folks,
>
> Thought I
Hi folks,
Thought I would ask here because it's somewhat confusing. I'm using Spark
2.4.5 on EMR 5.30.1 with Amazon MSK.
The version of Scala used is 2.11.12. I'm using this version of the
libraries spark-streaming-kafka-0-8_2.11-2.4.5.jar
Now I'm wanting to read from Kafka topics using Python
