[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974239#comment-14974239 ]
Steve Loughran commented on SPARK-11265: ---------------------------------------- What's changed? The spark code uses reflection to get the method {{("org.apache.hadoop.hive.ql.metadata.Hive#get"), then invokes it with a single argument: {{hive = hiveClass.getMethod("get").invoke(null, hiveConf.asInstanceOf[Object])}} Hive 0.13 has >1 method with this name, even in Hive 0.31.1; it has, in order, {{get(HiveConf}}, {{get(HiveConf, boolean)}}, and {{get()}}. Hive 1.2.1 adds one new method {{get(Configuration c, Class<?> clazz)}} *before* the others, and now invoke is failing as the returned method doesn't take a HiveConf. What could have been happening here is that the {{Class.get()}} method was returning the {{get(HiveConf}} method because it was first in the file, and on 1.2.1 the new method returned the new one, which didn't take a single {{HiveConf}}, hence the stack trace The fix, under all of it, is simply getting the method {{get(HiveConf.class)}}, and invoking it with the configuration created by reflection. That's all: explicitly asking for a method that's always been there. The code probably worked before just because nobody was looking at it. > YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster > --------------------------------------------------------------------- > > Key: SPARK-11265 > URL: https://issues.apache.org/jira/browse/SPARK-11265 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 1.5.1 > Environment: Kerberized Hadoop cluster > Reporter: Steve Loughran > > As reported on the dev list, trying to run a YARN client which wants to talk > to Hive in a Kerberized hadoop cluster fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org