[jira] [Comment Edited] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)

2015-08-31 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724500#comment-14724500
 ] 

Sergey Shelukhin edited comment on HIVE-11657 at 9/1/15 12:57 AM:
--

[~sushanth] can you take a look? https://reviews.apache.org/r/37980/


was (Author: sershe):
[~sushanth] can you take a look? 

> HIVE-2573 introduces some issues during metastore init (and CLI init)
> -
>
> Key: HIVE-11657
> URL: https://issues.apache.org/jira/browse/HIVE-11657
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-11657.patch
>
>
> HIVE-2573 introduced static reload functions call.
> It has a few problems:
> 1) When metastore client is initialized using an externally supplied config 
> (i.e. Hive.get(HiveConf)), it still gets called during static init using the 
> main service config. In my case, even though I have uris in the supplied 
> config to connect to remote MS (which eventually happens), the static call 
> creates objectstore, which is undesirable.
> 2) It breaks compat - old metastores do not support this call so new clients 
> will fail, and there's no workaround like not using a new feature because the 
> static call is always made



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (HIVE-11657) HIVE-2573 introduces some issues

2015-08-26 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715668#comment-14715668
 ] 

Sergey Shelukhin edited comment on HIVE-11657 at 8/26/15 10:50 PM:
---

[~navis] fyi if you have some easy fixes in mind :)

Example of logging I get 
{noformat}
2015-08-26 17:52:16,695 INFO [InputInitializer [Map 1] #0] 
metastore.HiveMetaStore: 0: Opening raw store with implemenation 
class:org.apache.hadoop.hive.metastore.ObjectStore
2015-08-26 17:52:16,730 INFO [InputInitializer [Map 1] #0] 
metastore.ObjectStore: ObjectStore, initialize called
[snip]
{noformat}

The call stack for the above is:
{noformat}
at 
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3054)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3074)
at 
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3299)
at 
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:175)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:167)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$MetastoreCache.getHive(OrcInputFormat.java:1544)
{noformat}
Then
{noformat}
2015-08-26 17:52:16,761 INFO [InputInitializer [Map 1] #0] hive.metastore: 
Trying to connect to metastore with URI thrift://[snip]:9085
2015-08-26 17:52:16,782 INFO [InputInitializer [Map 1] #0] hive.metastore: 
Opened a connection to metastore, current connections: 1
{noformat}



was (Author: sershe):
[~navis] fyi if you have some easy fixes in mind :)

Example of logging I get 
{noformat}
2015-08-26 17:52:16,695 INFO [InputInitializer [Map 1] #0] 
metastore.HiveMetaStore: 0: Opening raw store with implemenation 
class:org.apache.hadoop.hive.metastore.ObjectStore
2015-08-26 17:52:16,730 INFO [InputInitializer [Map 1] #0] 
metastore.ObjectStore: ObjectStore, initialize called
[snip]
{noformat}

The call stack for the above is:
{noformat}
at 
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3054)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3074)
at 
org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3299)
at 
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:175)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:167)
at 
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$MetastoreCache.getHive(OrcInputFormat.java:1544)
{noformat}
[snip]
{noformat}
2015-08-26 17:52:16,761 INFO [InputInitializer [Map 1] #0] hive.metastore: 
Trying to connect to metastore with URI thrift://[snip]:9085
2015-08-26 17:52:16,782 INFO [InputInitializer [Map 1] #0] hive.metastore: 
Opened a connection to metastore, current connections: 1
{noformat}


> HIVE-2573 introduces some issues
> 
>
> Key: HIVE-11657
> URL: https://issues.apache.org/jira/browse/HIVE-11657
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Critical
>
> HIVE-2573 introduced static reload functions call.
> It has a few problems:
> 1) When metastore client is initialized using an externally supplied config 
> (i.e. Hive.get(HiveConf)), it still gets called during static init using the 
> main service config. In my case, even though I have uris in the supplied 
> config to connect to remote MS (which eventually happens), the static call 
> creates objectstore, which is undesirable.
> 2) It breaks compat - old metastores do not support this call so new clients 
> will fail, and there's no workaround like not using a new feature because the 
> static call is always made



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)