Re: [E] Re: Pyspark Kafka Structured Stream not working.

2020-05-07 Thread Jungtaek Lim
com] > *Sent:* Wednesday, May 06, 2020 8:24 PM > *To:* Vijayant Kumar > *Cc:* user@spark.apache.org > *Subject:* [E] Re: Pyspark Kafka Structured Stream not working. > > > > *[EXTERNAL EMAIL]* DO NOT CLICK links or attachments unless you recognize > the sender and know the con

RE: [E] Re: Pyspark Kafka Structured Stream not working.

2020-05-07 Thread Vijayant Kumar
: [E] Re: Pyspark Kafka Structured Stream not working. [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. Hi, 1. You seem to use DStream (Spark Streaming), not Structured Streaming. 2. I'm not familiar with pyspark, but looks like

Re: Pyspark Kafka Structured Stream not working.

2020-05-06 Thread Jungtaek Lim
Hi, 1. You seem to use DStream (Spark Streaming), not Structured Streaming. 2. I'm not familiar with pyspark, but looks like the error message is very clear - Kafka doesn't allow such name for "client.id". The error message guides the naming rule, so you may need to be adopted with such

Pyspark Kafka Structured Stream not working.

2020-05-06 Thread Vijayant Kumar
Hi All, I am getting the below error while using Pyspark Structured Streaming from Kafka Producer. 20/05/06 11:51:16 ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - kafka.common.InvalidConfigException: client.id Python Kafka streamer is illegal, contains