I'm currently using Spark 1.4 in standalone mode. I've forked the Apache Hive branch from https://github.com/pwendell/hive <https://github.com/pwendell/hive> and customised in the following way.
Added a thread local variable in SessionManager class. And I'm setting the session variable in my Custom Authenticator class. For achieving the above, I've built the necessary jars(hive-common-0.13.1c.jar, hive-exec-0.13.1c.jar, hive-metastore-0.13.1c.jar, hive-serde-0.13.1c.jar, hive-service-0.13.1c.jar) from https://github.com/pwendell/hive and added to Spark's classpath. The above feature works in Spark 1.3.1, but is broken in Spark 1.4. When I looked into it, I found out that the ClientWrapper class is creating a new Session State and using it thereafter. As a result I'm not able to retrieve the info which i had stored earlier in the session. Also I'm not able to retrieve a value from hiveconf which was set earlier. When i looked into the source code for ClientWrapper.scala, i found the following. // Create an internal session state for this ClientWrapper. val state = { val original = Thread.currentThread().getContextClassLoader // Switch to the initClassLoader. Thread.currentThread().setContextClassLoader(initClassLoader) >From what i can understand, the above tries to use the existing Hive session, else it creates it's own session. Am I right? If so, is there a bug causing the ClientWrapper not to use the existing session. Or should I implement my requirement in a different way? My requirement is to have a custom session variable and use it throughout the session. My usage is as folows: To set the value /SessionManager.setSessionVar("value");/ To retrieve the value /SessionManager.getSessionVar();/ To set a hiveconf /hiveConf.set(conf, ConfVars.VAR, "val");/ to Retrieve /hiveConf.get(ConfVars.VAR); SessionState.get().getConf().getVar(ConfVars.VAR)/ -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hive-Session-gets-overwritten-in-ClientWrapper-tp23962.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org