Re: spark-submit parameters about two keytab files to yarn and kafka

2020-11-01 Thread kevin chen
Hi, Hope it can solve the issue by following method: *step 1 : * create a kafka kerberos config named kafka_client_jaas.conf: KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="./kafka.service.keytab" storeKey=true useTicketCache=false

Re: spark-submit parameters about two keytab files to yarn and kafka

2020-10-28 Thread Gabor Somogyi
Hi, Cross-realm trust must be configured. One can find several docs on how to do that. BR, G On Wed, Oct 28, 2020 at 8:21 AM big data wrote: > Hi, > > We want to submit spark streaming job to YARN and consume Kafka topic. > > YARN and Kafka are in two different clusters, and they have the >

spark-submit parameters about two keytab files to yarn and kafka

2020-10-28 Thread big data
Hi, We want to submit spark streaming job to YARN and consume Kafka topic. YARN and Kafka are in two different clusters, and they have the different kerberos authentication. We have two keytab files for YARN and Kafka. And my questions is how to add parameter for spark-submit command for