[
https://issues.apache.org/jira/browse/AMBARI-18013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated AMBARI-18013:
-----------------------------------------
Attachment: AMBARI-18013.patch
> HiveHook fails to post messages to kafka due to missing keytab config in
> /etc/hive/conf/atlas-application.properties in kerberized cluster
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-18013
> URL: https://issues.apache.org/jira/browse/AMBARI-18013
> Project: Ambari
> Issue Type: Bug
> Components: stacks
> Affects Versions: 2.4.0
> Reporter: Alejandro Fernandez
> Assignee: Alejandro Fernandez
> Fix For: 2.4.0
>
> Attachments: AMBARI-18013.patch
>
>
> STR:
> * Install Ambari 2.4
> * HDP 2.5 with Hive and Atlas
> * Kerberize the cluster
> The hive hook fails because 2 configs are missing from
> hive-atlas-application.properties,
> {noformat}
> atlas.jaas.KafkaClient.option.keyTab=/etc/security/keytabs/hive.service.keytab
> atlas.jaas.KafkaClient.option.principal=hive/_h...@example.com
> {noformat}
> *Impact: HiveHook related tests are failing.*
> {noformat}
> 2016-07-29 10:25:50,087 INFO [Atlas Logger 1]: producer.ProducerConfig
> (AbstractConfig.java:logAll(178)) - ProducerConfig values:
> metric.reporters = []
> metadata.max.age.ms = 300000
> reconnect.backoff.ms = 50
> sasl.kerberos.ticket.renew.window.factor = 0.8
> bootstrap.servers = [atlas-r6-bug-62789-1023re-2.openstacklocal:6667,
> atlas-r6-bug-62789-1023re-1.openstacklocal:6667]
> ssl.keystore.type = JKS
> sasl.mechanism = GSSAPI
> max.block.ms = 60000
> interceptor.classes = null
> ssl.truststore.password = null
> client.id =
> ssl.endpoint.identification.algorithm = null
> request.timeout.ms = 30000
> acks = 1
> receive.buffer.bytes = 32768
> ssl.truststore.type = JKS
> retries = 0
> ssl.truststore.location = null
> ssl.keystore.password = null
> send.buffer.bytes = 131072
> compression.type = none
> metadata.fetch.timeout.ms = 60000
> retry.backoff.ms = 100
> sasl.kerberos.kinit.cmd = /usr/bin/kinit
> buffer.memory = 33554432
> timeout.ms = 30000
> key.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
> sasl.kerberos.service.name = kafka
> sasl.kerberos.ticket.renew.jitter = 0.05
> ssl.trustmanager.algorithm = PKIX
> block.on.buffer.full = false
> ssl.key.password = null
> sasl.kerberos.min.time.before.relogin = 60000
> connections.max.idle.ms = 540000
> max.in.flight.requests.per.connection = 5
> metrics.num.samples = 2
> ssl.protocol = TLS
> ssl.provider = null
> ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
> batch.size = 16384
> ssl.keystore.location = null
> ssl.cipher.suites = null
> .protocol = PLAINTEXTSASL
> max.request.size = 1048576
> value.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
> ssl.keymanager.algorithm = SunX509
> metrics.sample.window.ms = 30000
> partitioner.class = class
> org.apache.kafka.clients.producer.internals.DefaultPartitioner
> linger.ms = 0
> 2016-07-29 10:25:50,091 INFO [Atlas Logger 1]: producer.KafkaProducer
> (KafkaProducer.java:close(658)) - Closing the Kafka producer with
> timeoutMillis = 0 ms.
> 2016-07-29 10:25:50,091 INFO [Atlas Logger 1]: hook.AtlasHook
> (AtlasHook.java:notifyEntitiesInternal(131)) - Failed to notify atlas for
> entity [[{Id='(type: hive_db, id: <unassigned>)', traits=[],
> values={owner=public, ownerType=2, qualifiedName=default@cl1,
> clusterName=cl1, name=default, description=Default Hive database,
> location=hdfs://atlas-r6-bug-62789-1023re-1.openstacklocal:8020/apps/hive/warehouse,
> parameters={}}}, {Id='(type: hive_table, id: <unassigned>)', traits=[],
> values={owner=hrt_qa, temporary=false, lastAccessTime=Fri Jul 29 10:25:49 UTC
> 2016, qualifiedName=default.t2@cl1, columns=[{Id='(type: hive_column, id:
> <unassigned>)', traits=[], values={owner=hrt_qa,
> qualifiedName=default.t2.abc@cl1, name=abc, comment=null, type=string,
> table=(type: hive_table, id: <unassigned>)}}], sd={Id='(type:
> hive_storagedesc, id: <unassigned>)', traits=[],
> values={qualifiedName=default.t2@cl1_storage, storedAsSubDirectories=false,
> location=hdfs://atlas-r6-bug-62789-1023re-1.openstacklocal:8020/apps/hive/warehouse/t2,
> compressed=false, inputFormat=org.apache.hadoop.mapred.TextInputFormat,
> outputFormat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
> parameters={}, serdeInfo=org.apache.atlas.typesystem.Struct@7648946d,
> table=(type: hive_table, id: <unassigned>), numBuckets=-1}},
> tableType=MANAGED_TABLE, createTime=Fri Jul 29 10:25:49 UTC 2016, name=t2,
> comment=null, partitionKeys=[], parameters={totalSize=0, numRows=0,
> rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, numFiles=0,
> transient_lastDdlTime=1469787949}, retention=0, db={Id='(type: hive_db, id:
> <unassigned>)', traits=[], values={owner=public, ownerType=2,
> qualifiedName=default@cl1, clusterName=cl1, name=default, description=Default
> Hive database,
> location=hdfs://atlas-r6-bug-62789-1023re-1.openstacklocal:8020/apps/hive/warehouse,
> parameters={}}}}}]]. Retrying
> org.apache.kafka.common.KafkaException: Failed to construct kafka producer
> at
> org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335)
> at
> org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188)
> at
> org.apache.atlas.kafka.KafkaNotification.createProducer(KafkaNotification.java:312)
> at
> org.apache.atlas.kafka.KafkaNotification.sendInternal(KafkaNotification.java:220)
> at
> org.apache.atlas.notification.AbstractNotification.send(AbstractNotification.java:84)
> at
> org.apache.atlas.hook.AtlasHook.notifyEntitiesInternal(AtlasHook.java:126)
> at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:111)
> at org.apache.atlas.hook.AtlasHook.notifyEntities(AtlasHook.java:157)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:274)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> javax..auth.login.LoginException: Could not login: the client is being asked
> for a password, but the Kafka client code does not currently support
> obtaining a password from the user. not available to garner authentication
> information from the user
> at
> org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
> at
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:71)
> at
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
> at
> org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277)
> ... 15 more
> Caused by: javax..auth.login.LoginException: Could not login: the client is
> being asked for a password, but the Kafka client code does not currently
> support obtaining a password from the user. not available to garner
> authentication information from the user
> at
> com.sun..auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940)
> at
> com.sun..auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at com.sun..auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax..auth.login.LoginContext.invoke(LoginContext.java:755)
> at javax..auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax..auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax..auth.login.LoginContext$4.run(LoginContext.java:680)
> at java..AccessController.doPrivileged(Native Method)
> at javax..auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax..auth.login.LoginContext.login(LoginContext.java:587)
> at
> org.apache.kafka.common..authenticator.AbstractLogin.login(AbstractLogin.java:69)
> at
> org.apache.kafka.common..kerberos.KerberosLogin.login(KerberosLogin.java:110)
> at
> org.apache.kafka.common..authenticator.LoginManager.<init>(LoginManager.java:46)
> at
> org.apache.kafka.common..authenticator.LoginManager.acquireLoginManager(LoginManager.java:68)
> at
> org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
> ... 18 more
> {noformat}
> This occurs because the Kerberos Descriptors does not allow using the same
> identity name more than once.
> AMBARI-17993 just made it possible to have unique names still reference the
> same identity via a "reference" tag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
