#general


@rayokota: @rayokota has joined the channel
@pabc55684: @pabc55684 has joined the channel
@pabc55684: Hi Everyone, I am trying to integrate Kerberos Hadoop with Pinot.and using below configurations. Executables: export HADOOP_HOME=/usr/hdp/2.6.3.0-235/hadoop export HADOOP_VERSION=2.7.3.2.6.3.0-235 export HADOOP_GUAVA_VERSION=11.0.2 export HADOOP_GSON_VERSION=2.2.4 export GC_LOG_LOCATION=/home/hdfs/Pinot/pinotGcLog export PINOT_VERSION=0.7.1 export PINOT_DISTRIBUTION_DIR=/home/hdfs/apache-pinot-incubating-0.7.1-bin export HADOOP_CLIENT_OPTS="-Dplugins.dir=${PINOT_DISTRIBUTION_DIR}/plugins -Dlog4j2.configurationFile=${PINOT_DISTRIBUTION_DIR}/conf/pinot-ingestion-job-log4j2.xml" export SERVER_CONF_DIR=/home/hdfs/apache-pinot-incubating-0.7.1-bin/bin export ZOOKEEPER_ADDRESS=<ZOOKEEPER_ADDRESS> export CLASSPATH_PREFIX="${HADOOP_HOME}/hadoop-hdfs/hadoop-hdfs-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-annotations-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-auth-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-common-${HADOOP_VERSION}.jar:${HADOOP_HOME}/lib/guava-${HADOOP_GUAVA_VERSION}.jar:${HADOOP_HOME}/lib/gson-${HADOOP_GSON_VERSION}.jar" export JAVA_OPTS="-Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:${GC_LOG_LOCATION}/gc-pinot-server.log" controller.conf controller.data.dir=<fs.defaultFS>/user/hdfs/controller_segment controller.local.temp.dir=/home/hdfs/Pinot/pinot_tmp/ controller.zk.str=<ZOOKEEPER_ADDRESS> controller.enable.split.commit=true controller.access.protocols.http.port=9000 controller.helix.cluster.name=PinotCluster pinot.controller.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS pinot.controller.storage.factory.hdfs.hadoop.conf.path=/usr/hdp/2.6.3.0-235/hadoop/conf pinot.controller.segment.fetcher.protocols=file,http,hdfs pinot.controller.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.principle='h...@hdfssithdp.com' pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.keytab='/home/hdfs/hdfs.keytab' pinot.controller.storage.factory.hdfs.hadoop.kerberos.principle='h...@hdfssithdp.com' pinot.controller.storage.factory.hdfs.hadoop.kerberos.keytab='/home/hdfs/hdfs.keytab' controller.vip.port=9000 controller.port=9000 pinot.set.instance.id.to.hostname=true pinot.server.grpc.enable=true Kerbeous Information: kinit -V -k -t /home/hdfs/hdfs.keytab h...@hdfssithdp.com Using default cache: /tmp/krb5cc_57372 Using principal: h...@hdfssithdp.com Using keytab: /home/hdfs/hdfs.keytab Authenticated to Kerberos v5 ERROR MESSAGE: END: Invoking TASK controller pipeline for event ResourceConfigChange::15fc3764_TASK for cluster PinotCluster, took 278 ms START AsyncProcess: TASK::TaskGarbageCollectionStage END AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Trying to authenticate user 'h...@hdfssithdp.com' with keytab '/home/hdfs/hdfs.keytab'.. Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs java.lang.RuntimeException: Failed to authenticate user principal ['h...@hdfssithdp.com'] with keytab ['/home/hdfs/hdfs.keytab'] at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:258) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]     Caused by: java.io.IOException: Login failure for 'h...@hdfssithdp.com' from keytab '/home/hdfs/hdfs.keytab': javax.security.auth.login.LoginException: Unable to obtain password from user. at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:962) ~[pinot-orc-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6] at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:254) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6] ... 15 more     Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:901) ~[?:1.8.0_241] at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764) ~[?:1.8.0_241] at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) ~[?:1.8.0_241] at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:254) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6] ... 15 more     Failed to start a Pinot [CONTROLLER] at 21.954 since launch     java.lang.RuntimeException: java.lang.RuntimeException: Failed to authenticate user principal ['h...@hdfssithdp.com'] with keytab ['/home/hdfs/hdfs.keytab'] at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6] P.s. I am executing this hdfs user and for keytab file also user is hdfs .I have also given 777 access to hdfs.keytab file. someone Kindly suggest What is the issue here.I have read multiple blocks and everywhere found that it is because of wrong prinicpal/keytab file combination/user don't have access/give 777 access to file/try with different user. tried all the the options but nothing worked as of now.
  @xiangfu0: is this same thread as:
@r.clark: @r.clark has joined the channel

#random


@rayokota: @rayokota has joined the channel
@pabc55684: @pabc55684 has joined the channel
@r.clark: @r.clark has joined the channel

#troubleshooting


@rayokota: @rayokota has joined the channel
@pabc55684: @pabc55684 has joined the channel
@r.clark: @r.clark has joined the channel

#pinot-dev


@atri.sharma: @atri.sharma has joined the channel
@mertaydin: @mertaydin has joined the channel

#announcements


@dbarber: @dbarber has joined the channel

#presto-pinot-streaming


@atri.sharma: @atri.sharma has joined the channel

#getting-started


@satyamkrishna92: @satyamkrishna92 has joined the channel
@mertaydin: @mertaydin has joined the channel
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org

Reply via email to