[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715601#comment-14715601 ] Sergey Shelukhin commented on HIVE-11657: - [~gopalv] fyi... I might take a look at (1) at some point, will probably fix (2) with it, unless something happens earlier > HIVE-2573 introduces some issues > > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Critical > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config, > it still gets called during static init using the main service config. In my > case, even though I have uris in the supplied config to connect to remote MS > (which eventually happens), the static call creates objectstore, which is > undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715668#comment-14715668 ] Sergey Shelukhin commented on HIVE-11657: - [~navis] fyi if you have some easy fixes in mind :) Example of logging I get {noformat} 2015-08-26 17:52:16,695 INFO [InputInitializer [Map 1] #0] metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2015-08-26 17:52:16,730 INFO [InputInitializer [Map 1] #0] metastore.ObjectStore: ObjectStore, initialize called [snip] {noformat} The call stack for the above is: {noformat} at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3054) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3074) at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3299) at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:175) at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:167) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$MetastoreCache.getHive(OrcInputFormat.java:1544) {noformat} [snip] {noformat} 2015-08-26 17:52:16,761 INFO [InputInitializer [Map 1] #0] hive.metastore: Trying to connect to metastore with URI thrift://[snip]:9085 2015-08-26 17:52:16,782 INFO [InputInitializer [Map 1] #0] hive.metastore: Opened a connection to metastore, current connections: 1 {noformat} > HIVE-2573 introduces some issues > > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Critical > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717877#comment-14717877 ] Sergey Shelukhin commented on HIVE-11657: - I will take a look tomorrow. This is causing annoying and unobvious problems on some setups. > HIVE-2573 introduces some issues > > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721006#comment-14721006 ] Hive QA commented on HIVE-11657: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12753084/HIVE-11657.patch {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9380 tests executed *Failed tests:* {noformat} org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation org.apache.hive.hcatalog.streaming.TestStreaming.testRemainingTransactions {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5111/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5111/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5111/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12753084 - PreCommit-HIVE-TRUNK-Build > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724500#comment-14724500 ] Sergey Shelukhin commented on HIVE-11657: - [~sushanth] can you take a look? > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725024#comment-14725024 ] Navis commented on HIVE-11657: -- [~sershe] Making static call in Hive was one of the my worst decision on hive. I was so tired of rebasing the patch for years and new things like permanent function bothered too much of me and I couldn't think of any better idea in that day. > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725778#comment-14725778 ] Sergey Shelukhin commented on HIVE-11657: - [~navis] np :) can you review the change? > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11657) HIVE-2573 introduces some issues during metastore init (and CLI init)
[ https://issues.apache.org/jira/browse/HIVE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727709#comment-14727709 ] Sushanth Sowmyan commented on HIVE-11657: - +1. > HIVE-2573 introduces some issues during metastore init (and CLI init) > - > > Key: HIVE-11657 > URL: https://issues.apache.org/jira/browse/HIVE-11657 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: HIVE-11657.patch > > > HIVE-2573 introduced static reload functions call. > It has a few problems: > 1) When metastore client is initialized using an externally supplied config > (i.e. Hive.get(HiveConf)), it still gets called during static init using the > main service config. In my case, even though I have uris in the supplied > config to connect to remote MS (which eventually happens), the static call > creates objectstore, which is undesirable. > 2) It breaks compat - old metastores do not support this call so new clients > will fail, and there's no workaround like not using a new feature because the > static call is always made -- This message was sent by Atlassian JIRA (v6.3.4#6332)