[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989368#comment-14989368 ] Apache Spark commented on SPARK-11265: -- User 'steveloughran' has created a pull request for this issue: https://github.com/apache/spark/pull/9466 > YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster > - > > Key: SPARK-11265 > URL: https://issues.apache.org/jira/browse/SPARK-11265 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.5.1 > Environment: Kerberized Hadoop cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 1.6.0 > > > As reported on the dev list, trying to run a YARN client which wants to talk > to Hive in a Kerberized hadoop cluster fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987583#comment-14987583 ] Apache Spark commented on SPARK-11265: -- User 'steveloughran' has created a pull request for this issue: https://github.com/apache/spark/pull/9438 > YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster > - > > Key: SPARK-11265 > URL: https://issues.apache.org/jira/browse/SPARK-11265 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.5.1 > Environment: Kerberized Hadoop cluster >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 1.6.0 > > > As reported on the dev list, trying to run a YARN client which wants to talk > to Hive in a Kerberized hadoop cluster fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974239#comment-14974239 ] Steve Loughran commented on SPARK-11265: What's changed? The spark code uses reflection to get the method {{("org.apache.hadoop.hive.ql.metadata.Hive#get"), then invokes it with a single argument: {{hive = hiveClass.getMethod("get").invoke(null, hiveConf.asInstanceOf[Object])}} Hive 0.13 has >1 method with this name, even in Hive 0.31.1; it has, in order, {{get(HiveConf}}, {{get(HiveConf, boolean)}}, and {{get()}}. Hive 1.2.1 adds one new method {{get(Configuration c, Class clazz)}} *before* the others, and now invoke is failing as the returned method doesn't take a HiveConf. What could have been happening here is that the {{Class.get()}} method was returning the {{get(HiveConf}} method because it was first in the file, and on 1.2.1 the new method returned the new one, which didn't take a single {{HiveConf}}, hence the stack trace The fix, under all of it, is simply getting the method {{get(HiveConf.class)}}, and invoking it with the configuration created by reflection. That's all: explicitly asking for a method that's always been there. The code probably worked before just because nobody was looking at it. > YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster > - > > Key: SPARK-11265 > URL: https://issues.apache.org/jira/browse/SPARK-11265 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.5.1 > Environment: Kerberized Hadoop cluster >Reporter: Steve Loughran > > As reported on the dev list, trying to run a YARN client which wants to talk > to Hive in a Kerberized hadoop cluster fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org