Hi Darshan,
How are you creating your kafka stream? Can you please share the options
you provide?
spark.readStream.format("kafka")
.option(...) // all these please
.load()
On Sat, Oct 14, 2017 at 1:55 AM, Darshan Pandya <darshanpan...@gmail.com>
wrote:
> Hello,
>
> I'm using Spark 2.1.0 on CDH 5.8 with kafka 0.10.0.1 + kerberos
>
> I am unable to connect to the kafka broker with the following message
>
>
> 17/10/14 14:29:10 WARN clients.NetworkClient: Bootstrap broker
> 10.197.19.25:9092 disconnected
>
> and is unable to consume any messages.
>
> And am using it as follows
>
> jaas.conf
>
> KafkaClient {
> com.sun.security.auth.module.Krb5LoginModule required
> useKeyTab=true
> keyTab="./gandalf.keytab"
> storeKey=true
> useTicketCache=false
> serviceName="kafka"
> principal="gand...@domain.com";
> };
>
> $SPARK_HOME/bin/spark-submit \
> --master yarn \
> --files jaas.conf,gandalf.keytab \
> --driver-java-options "-Djava.security.auth.login.config=./jaas.conf
> -Dhdp.version=2.4.2.0-258" \
> --conf
> "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./jaas.conf"
> \
> --class com.example.ClassName uber-jar-with-deps-and-hive-site.jar
>
> Thanks in advance.
>
>
> --
> Sincerely,
> Darshan
>
>
