Well, looks like a huge coincidence, but this was just sent to github: https://github.com/apache/spark/pull/2967
On Mon, Oct 27, 2014 at 3:25 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > Hey guys, > > I've been using the "HiveFromSpark" example to test some changes and I > ran into an issue that manifests itself as an NPE inside Hive code > because some configuration object is null. > > Tracing back, it seems that `sessionState` being a lazy val in > HiveContext is causing it. That variably is only evaluated in [1], > while the call in [2] causes a Driver to be initialized by [3], which > the tries to use the thread-local session state ([4]) which hasn't > been set yet. > > This could be seen as a Hive bug ([3] should probably be calling the > constructor that takes a conf object), but is there a reason why these > fields are lazy in HiveContext? I explicitly called > SessionState.setCurrentSessionState() before the > CommandProcessorFactory call and that seems to fix the issue too. > > [1] > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala#L305 > [2] > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala#L289 > [3] > https://github.com/apache/hive/blob/9c63b2fdc35387d735f4c9d08761203711d4974b/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandProcessorFactory.java#L104 > [4] > https://github.com/apache/hive/blob/9c63b2fdc35387d735f4c9d08761203711d4974b/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L286 > > -- > Marcelo -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
