I am working on a spark job which reads from kafka topic and write to HDFS however while submitting the job using spark-submit command I am getting following error.
Error log Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. Make sure -Djava.security.auth.login.config property passed to JVM and the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)'. Make sure you are using FQDN of the Kafka broker you are trying to connect to. not available to garner authentication information from the user I am passing the user keytab and kafka_client_jaas.conf file to spark submit command as suggested in Hortonworks documentation or https://github.com/hortonworks-spark/skc#running-on-a-kerberos-enabled-cluster Passing following parameters to spark-submit --files user.keytab,kafka_client_jaas.conf \ --driver-java-options "-Djava.security.auth.login.config=kafka_client_jaas.conf" \ --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=kafka_client_jaas.conf" \ Version information Spark version - 2.2.0.2.6.4.0-91 Kafka version - 0.10.1 Any help is much appreciated. Thanks, Amol
