Thanks Prasad, patch is available in HIVE-816.
On 9/4/09 4:24 PM, "Prasad Chakka" <pcha...@facebook.com> wrote: > Yeah, you are correct. It is a bug. It will be great if you can submit a > patch. > > Prasad > > > > From: David Lerman <dler...@videoegg.com> > Reply-To: <hive-user@hadoop.apache.org> > Date: Fri, 4 Sep 2009 13:19:28 -0700 > To: <hive-user@hadoop.apache.org> > Subject: MetaStoreClient not getting cached > > Planning a query is taking a very long time, and during the pause, I see the > following log lines hundreds of times in a row > > 2009-09-04 16:14:17,090 INFO metastore.HiveMetaStore > (HiveMetaStore.java:getMS(125)) - 0: Opening raw store with implemenation > class:org.apache.hadoop.hive.metastore.ObjectStore > 2009-09-04 16:14:17,090 > INFO metastore.ObjectStore (ObjectStore.java:initialize(120)) - > ObjectStore, initialize called > 2009-09-04 16:14:17,090 INFO > metastore.ObjectStore (ObjectStore.java:setConf(106)) - Initialized > ObjectStore > > Going through the code, I see that the PartitionPruner is asking for > metadata on every partition, and every call ends up calling > org.apache.hadoop.hive.ql.metadata.Hive.getMSC(): > > private IMetaStoreClient getMSC() throws MetaException { > IMetaStoreClient msc = threadLocalMSC.get(); > if(msc == null) { > msc = this.createMetaStoreClient(); > } > return msc; > } > > Each call ends up creating a new instance, because threadLocalMSC.set() is > never called. Is there a reason this isn't: > > private IMetaStoreClient getMSC() throws MetaException { > IMetaStoreClient msc = threadLocalMSC.get(); > if(msc == null) { > msc = this.createMetaStoreClient(); > threadLocalMSC.set(msc); // CHANGE HERE! > } > return msc; > } > > Thanks!