[
https://issues.apache.org/jira/browse/LIVY-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicola Marangoni updated LIVY-700:
----------------------------------
Environment: Cloudera 5.7 Hadoop 2.6 Spark 2.4.x Centos7 Livy 0.6 Zeppelin
0.8.2 (was: Cloudera 5.7 Hadoop 2.6 Spark 2.4.x Centos7 Livy 0.6 Zeppelin 8.2)
> Connection refused from YARN when impersonation enabled
> -------------------------------------------------------
>
> Key: LIVY-700
> URL: https://issues.apache.org/jira/browse/LIVY-700
> Project: Livy
> Issue Type: Bug
> Components: Interpreter
> Affects Versions: 0.6.0
> Environment: Cloudera 5.7 Hadoop 2.6 Spark 2.4.x Centos7 Livy 0.6
> Zeppelin 0.8.2
> Reporter: Nicola Marangoni
> Priority: Major
> Labels: YARN, impersonation
>
> I have Livy and Zeppelin running under the user "zeppelin" on an edge node.
> The Cloudera-cluster is not kerberized. Livy is set in yarn client mode.
> When the impersonation is not enabled the Livy interpreter works, and a YARN
> job ist started on the cluster using the user "zeppelin".
> When *livy.impersonation.enabled = true* Livy cannot connect to the YARN RM:
> {code:java}
> 19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO
> client.ConfiguredRMFailoverProxyProvider: Failing over to rm271
> 19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO
> retry.RetryInvocationHandler: Exception while invoking getClusterMetrics of
> class ApplicationClientProtocolPBClientImpl over rm271 after 1 fail over
> attempts. Trying to fail over immediately.
> 19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO
> client.ConfiguredRMFailoverProxyProvider: Failing over to rm270
> 19/10/24 14:40:58 INFO utils.LineBufferedStream: 19/10/24 14:40:58 INFO
> retry.RetryInvocationHandler: Exception while invoking getClusterMetrics of
> class ApplicationClientProtocolPBClientImpl over rm270 after 2 fail over
> attempts. Trying to fail over after sleeping for 374ms.
> 19/10/24 14:40:58 INFO utils.LineBufferedStream: java.net.ConnectException:
> Call From <localhost>/123.123.123.123 to yarn.rm.host.name:8032 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused{code}
> This error happen after the spark driver is successfully started.
> Spark simply doesn't get any resources from YARN.
> Zeppelin is configured to use FreeIPA over LDAP for login. The user I use to
> login to the zeppelin UI has even more rights on the cluster and if I start a
> spark-shell on the same cluster with that user I get a YARN job without any
> issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
