Thanks for your reply. Michael: No. I only create one HiveContext in the code.
Hao: Yes. I subclass HiveContext and defines own function to create database and then subclass akka Actor to call that function in response to an abstract message. By your suggestion, I called println(sessionState.getConf.getAllProperties) that printed tons of properties; however, the same NullPointerException was still thrown. As mentioned, the weird thing is that everything worked fine if I simply called actor.hiveContext.createDB() directly. But it throws the null pointer exception from Driver.java if I do "actor ! CreateSomeDB”, which seems to me just the same thing because the actor does nothing but call createDB(). Du From: Michael Armbrust <mich...@databricks.com<mailto:mich...@databricks.com>> Date: Wednesday, September 17, 2014 at 7:40 PM To: "Cheng, Hao" <hao.ch...@intel.com<mailto:hao.ch...@intel.com>> Cc: Du Li <l...@yahoo-inc.com.invalid<mailto:l...@yahoo-inc.com.invalid>>, "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: problem with HiveContext inside Actor - dev Is it possible that you are constructing more than one HiveContext in a single JVM? Due to global state in Hive code this is not allowed. Michael On Wed, Sep 17, 2014 at 7:21 PM, Cheng, Hao <hao.ch...@intel.com<mailto:hao.ch...@intel.com>> wrote: Hi, Du I am not sure what you mean “triggers the HiveContext to create a database”, do you create the sub class of HiveContext? Just be sure you call the “HiveContext.sessionState” eagerly, since it will set the proper “hiveconf” into the SessionState, otherwise the HiveDriver will always get the null value when retrieving HiveConf. Cheng Hao From: Du Li [mailto:l...@yahoo-inc.com.INVALID<mailto:l...@yahoo-inc.com.INVALID>] Sent: Thursday, September 18, 2014 7:51 AM To: user@spark.apache.org<mailto:user@spark.apache.org>; d...@spark.apache.org<mailto:d...@spark.apache.org> Subject: problem with HiveContext inside Actor Hi, Wonder anybody had similar experience or any suggestion here. I have an akka Actor that processes database requests in high-level messages. Inside this Actor, it creates a HiveContext object that does the actual db work. The main thread creates the needed SparkContext and passes in to the Actor to create the HiveContext. When a message is sent to the Actor, it is processed properly except that, when the message triggers the HiveContext to create a database, it throws a NullPointerException in hive.ql.Driver.java which suggests that its conf variable is not initialized. Ironically, it works fine if my main thread directly calls actor.hiveContext to create the database. The spark version is 1.1.0. Thanks, Du
