Hi Jungtek, Thanks for the response. It appears to be #1. I will appreciate if you can share some sample command to submit the Spark application.?
From: Jungtaek Lim [mailto:kabhwan.opensou...@gmail.com] Sent: Wednesday, May 06, 2020 8:24 PM To: Vijayant Kumar <vijayant.ku...@mavenir.com.invalid> Cc: user@spark.apache.org Subject: [E] Re: Pyspark Kafka Structured Stream not working. [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. Hi, 1. You seem to use DStream (Spark Streaming), not Structured Streaming. 2. I'm not familiar with pyspark, but looks like the error message is very clear - Kafka doesn't allow such name for "client.id<http://client.id>". The error message guides the naming rule, so you may need to be adopted with such convention. (e.g. no space) Hope this helps, Thanks, Jungtaek Lim (HeartSaVioR) On Wed, May 6, 2020 at 5:36 PM Vijayant Kumar <vijayant.ku...@mavenir.com.invalid<mailto:vijayant.ku...@mavenir.com.invalid>> wrote: Hi All, I am getting the below error while using Pyspark Structured Streaming from Kafka Producer. 20/05/06 11:51:16 ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - kafka.common.InvalidConfigException: client.id<http://client.id> Python Kafka streamer is illegal, contains a character other than ASCII alphanumerics, '.', '_' and '-' I am using the below code to get the messages: broker='vm105:2181' topic='Hello-Kafka' print 'broker topic is ',broker,topic kvs = KafkaUtils.createStream(ssc, \ broker, \ "Python Kafka streamer",{topic:1}) And my Submit command is like below :- spark-submit --jars spark-streaming-kafka-0-8-assembly_2.11-2.4.4.jar test_kafka.py vm105:2181 Hello-Kafka Can any one help me what am I missing. ? Thanks, Vijayant ________________________________ This e-mail message may contain confidential or proprietary information of Mavenir Systems, Inc. or its affiliates and is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies in your control and contact us by e-mailing to secur...@mavenir.com<mailto:secur...@mavenir.com>. This message contains the views of its author and may not necessarily reflect the views of Mavenir Systems, Inc. or its affiliates, who employ systems to monitor email messages, but make no representation that such messages are authorized, secure, uncompromised, or free from computer viruses, malware, or other defects. Thank You ________________________________ This e-mail message may contain confidential or proprietary information of Mavenir Systems, Inc. or its affiliates and is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies in your control and contact us by e-mailing to secur...@mavenir.com. This message contains the views of its author and may not necessarily reflect the views of Mavenir Systems, Inc. or its affiliates, who employ systems to monitor email messages, but make no representation that such messages are authorized, secure, uncompromised, or free from computer viruses, malware, or other defects. Thank You
