Thanks David. This will be in trunk & 0.4 soon.
Prasad
________________________________
From: David Lerman <dler...@videoegg.com>
Reply-To: <hive-user@hadoop.apache.org>
Date: Fri, 4 Sep 2009 13:41:43 -0700
To: <hive-user@hadoop.apache.org>
Subject: Re: MetaStoreClient not getting cached
Thanks Prasad, patch is available in HIVE-816.
On 9/4/09 4:24 PM, "Prasad Chakka" <pcha...@facebook.com> wrote:
> Yeah, you are correct. It is a bug. It will be great if you can submit a
> patch.
>
> Prasad
>
>
>
> From: David Lerman <dler...@videoegg.com>
> Reply-To: <hive-user@hadoop.apache.org>
> Date: Fri, 4 Sep 2009 13:19:28 -0700
> To: <hive-user@hadoop.apache.org>
> Subject: MetaStoreClient not getting cached
>
> Planning a query is taking a very long time, and during the pause, I see the
> following log lines hundreds of times in a row
>
> 2009-09-04 16:14:17,090 INFO metastore.HiveMetaStore
> (HiveMetaStore.java:getMS(125)) - 0: Opening raw store with implemenation
> class:org.apache.hadoop.hive.metastore.ObjectStore
> 2009-09-04 16:14:17,090
> INFO metastore.ObjectStore (ObjectStore.java:initialize(120)) -
> ObjectStore, initialize called
> 2009-09-04 16:14:17,090 INFO
> metastore.ObjectStore (ObjectStore.java:setConf(106)) - Initialized
> ObjectStore
>
> Going through the code, I see that the PartitionPruner is asking for
> metadata on every partition, and every call ends up calling
> org.apache.hadoop.hive.ql.metadata.Hive.getMSC():
>
> private IMetaStoreClient getMSC() throws MetaException {
> IMetaStoreClient msc = threadLocalMSC.get();
> if(msc == null) {
> msc = this.createMetaStoreClient();
> }
> return msc;
> }
>
> Each call ends up creating a new instance, because threadLocalMSC.set() is
> never called. Is there a reason this isn't:
>
> private IMetaStoreClient getMSC() throws MetaException {
> IMetaStoreClient msc = threadLocalMSC.get();
> if(msc == null) {
> msc = this.createMetaStoreClient();
> threadLocalMSC.set(msc); // CHANGE HERE!
> }
> return msc;
> }
>
> Thanks!
