It's not either 1 or 2. Both two items are applied. I haven't played with DStream + pyspark but given the error message is clear you'll probably want to change the client.id "Python Kafka streamer" to accommodate the naming convention guided in error message.
On Thu, May 7, 2020 at 3:55 PM Vijayant Kumar <vijayant.ku...@mavenir.com> wrote: > Hi Jungtek, > > > > Thanks for the response. It appears to be #1. > > I will appreciate if you can share some sample command to submit the Spark > application.? > > > > *From:* Jungtaek Lim [mailto:kabhwan.opensou...@gmail.com] > *Sent:* Wednesday, May 06, 2020 8:24 PM > *To:* Vijayant Kumar <vijayant.ku...@mavenir.com.invalid> > *Cc:* user@spark.apache.org > *Subject:* [E] Re: Pyspark Kafka Structured Stream not working. > > > > *[EXTERNAL EMAIL]* DO NOT CLICK links or attachments unless you recognize > the sender and know the content is safe. > > Hi, > > > > 1. You seem to use DStream (Spark Streaming), not Structured Streaming. > > 2. I'm not familiar with pyspark, but looks like the error message is very > clear - Kafka doesn't allow such name for "client.id". The error message > guides the naming rule, so you may need to be adopted with such convention. > (e.g. no space) > > > > Hope this helps, > > > > Thanks, > > Jungtaek Lim (HeartSaVioR) > > > > On Wed, May 6, 2020 at 5:36 PM Vijayant Kumar < > vijayant.ku...@mavenir.com.invalid> wrote: > > Hi All, > > > > I am getting the below error while using Pyspark Structured Streaming from > Kafka Producer. > > > > 20/05/06 11:51:16 ERROR ReceiverTracker: Deregistered receiver for stream > 0: Error starting receiver 0 - kafka.common.InvalidConfigException: > client.id Python Kafka streamer is illegal, contains a character other > than ASCII alphanumerics, '.', '_' and '-' > > > > I am using the below code to get the messages: > > > > broker='vm105:2181' > > topic='Hello-Kafka' > > print 'broker topic is ',broker,topic > > kvs = KafkaUtils.createStream(ssc, \ > > broker, \ > > "Python Kafka streamer",{topic:1}) > > > > And my Submit command is like below :- > > *spark-submit --jars spark-streaming-kafka-0-8-assembly_2.11-2.4.4.jar > test_kafka.py vm105:2181 Hello-Kafka* > > > > Can any one help me what am I missing. ? > > > > Thanks, > > Vijayant > ------------------------------ > > This e-mail message may contain confidential or proprietary information of > Mavenir Systems, Inc. or its affiliates and is intended solely for the use > of the intended recipient(s). If you are not the intended recipient of this > message, you are hereby notified that any review, use or distribution of > this information is absolutely prohibited and we request that you delete > all copies in your control and contact us by e-mailing to > secur...@mavenir.com. This message contains the views of its author and > may not necessarily reflect the views of Mavenir Systems, Inc. or its > affiliates, who employ systems to monitor email messages, but make no > representation that such messages are authorized, secure, uncompromised, or > free from computer viruses, malware, or other defects. Thank You > > ------------------------------ > > This e-mail message may contain confidential or proprietary information of > Mavenir Systems, Inc. or its affiliates and is intended solely for the use > of the intended recipient(s). If you are not the intended recipient of this > message, you are hereby notified that any review, use or distribution of > this information is absolutely prohibited and we request that you delete > all copies in your control and contact us by e-mailing to > secur...@mavenir.com. This message contains the views of its author and > may not necessarily reflect the views of Mavenir Systems, Inc. or its > affiliates, who employ systems to monitor email messages, but make no > representation that such messages are authorized, secure, uncompromised, or > free from computer viruses, malware, or other defects. Thank You >
