[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13547934#comment-13547934 ] Hudson commented on HIVE-2585: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-2585 Collapse hive.metastore.uris and hive.metastore.local (Ashutosh Chauhan via egc) (Revision 1352333) Result = ABORTED ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352333 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/conf/hive-site.xml * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreAuthorization.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java * /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.1.patch, > ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.2.patch, hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472714#comment-13472714 ] Ashutosh Chauhan commented on HIVE-2585: Sounds good. Please file an issue and patch. > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.1.patch, > ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.2.patch, hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472636#comment-13472636 ] Travis Crawford commented on HIVE-2585: --- Hive clients using a remote metastore via {{hive.metastore.uris}} works correctly, however, this error message is always printed because AFAICT its not possible null out {{javax.jdo.option.ConnectionURL}} from {{hive-site.xml}}. Looking through the comments on this patch, the error message was intended to reduce user confusion, a goal I totally support. In [HiveMetaStoreClient.java|https://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java] we see a very clear info message printed saying that a remote metastore is used. {code:title=HiveMetaStoreClient.java logs that a remote metastore is used} LOG.info("Trying to connect to metastore with URI " + store); ... LOG.info("Connected to metastore."); {code} Since we clearly communicate to the user that a remote metastore at the given URI is being used, is this warning needed? I'd be happy to submit a patch that removes the above warning message, and makes the following HiveMetaStoreClient logging change: {code} LOG.debug("Trying to connect to remote HiveMetaStore: " + store); ... LOG.info("Connected to remote HiveMetaStore: " + store); {code} The change is at debug level we print connection attempts, and always print which remote HiveMetaStore we actually connected to. Thoughts? If this sounds good I'll file a related issue and submit the patch. > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.1.patch, > ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.2.patch, hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472595#comment-13472595 ] Ashutosh Chauhan commented on HIVE-2585: Is it just that you are seeing a log message, which you should not have been seeing or is it affecting some functionality as well. > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.1.patch, > ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.2.patch, hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472578#comment-13472578 ] Travis Crawford commented on HIVE-2585: --- I started seeing this error message in my logs, and when investigating believe there's an issue here. AFAICT, if you set {{hive.metastore.uris}} you will ALWAYS see this error. The reason is {{javax.jdo.option.ConnectionURL}} has a default value and will never be null. I set this property in {{hive-site.xml}} and walked through the configuration loading in a debugger. If the value is not empty it takes effect, and is ignored if empty. Since {{javax.jdo.option.ConnectionURL}} has a default and cannot be unset, this warning will always be printed if someone sets {{hive.metastore.uris}}. Before diving into possible solutions, [~ashutoshc] & [~appodictic] can you confirm this is an issue, or clarify how to correctly configure this? If this is a user error I can post a patch clarifying the error message so others that see this message will know what to do. {code} javax.jdo.option.ConnectionURL {code} {code} +if (null != this.get(ConfVars.METASTOREURIS.varname, null) && +null != this.get(ConfVars.METASTORECONNECTURLKEY.varname, null)) { + l4j.error("Found both " + ConfVars.METASTOREURIS.varname + " and " + +ConfVars.METASTORECONNECTURLKEY + " Recommended to have exactly one of those config key" + +"in configuration"); +} {code} > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.1.patch, > ASF.LICENSE.NOT.GRANTED--HIVE-2585.D2559.2.patch, hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398231#comment-13398231 ] Hudson commented on HIVE-2585: -- Integrated in Hive-trunk-h0.21 #1501 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1501/]) HIVE-2585 Collapse hive.metastore.uris and hive.metastore.local (Ashutosh Chauhan via egc) (Revision 1352333) Result = SUCCESS ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352333 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/conf/hive-site.xml * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartitionRemote.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreAuthorization.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java * /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 0.10.0 > > Attachments: HIVE-2585.D2559.1.patch, HIVE-2585.D2559.2.patch, > hive-2585_3.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397816#comment-13397816 ] Edward Capriolo commented on HIVE-2585: --- +1 On changes can you do one more rebase and will commit. {noformat} [edward@tablitha trunk]$ patch -p0 --dry-run < HIVE-2585.D2559.2.patch patching file metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java Hunk #1 FAILED at 91. 1 out of 1 hunk FAILED -- saving rejects to file metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStoreIpAddress.java.rej {noformat} > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch, HIVE-2585.D2559.2.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280468#comment-13280468 ] Ashutosh Chauhan commented on HIVE-2585: Bump. Got hit by this today again. Carl, do you have any further suggestions. Else, I think its ready to go in? > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch, HIVE-2585.D2559.2.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248781#comment-13248781 ] Phabricator commented on HIVE-2585: --- ashutoshc has commented on the revision "HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local". INLINE COMMENTS conf/hive-default.xml.template:102 Valid point. Then, instead of throwing exception, we can log a message then. REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247885#comment-13247885 ] Phabricator commented on HIVE-2585: --- thw has commented on the revision "HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local". INLINE COMMENTS conf/hive-default.xml.template:102 A scenario where this would lead to change of existing behavior is when client and metastore server point to the same conf, with client using the thrift URI and server javax.jdo.option.ConnectionURL (should only happen in dev environments). REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247875#comment-13247875 ] Phabricator commented on HIVE-2585: --- cwsteinbach has commented on the revision "HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local". INLINE COMMENTS conf/hive-default.xml.template:102 HiveConf.init() seems like the right place to make these checks. REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246491#comment-13246491 ] Phabricator commented on HIVE-2585: --- ashutoshc has commented on the revision "HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local". INLINE COMMENTS conf/hive-default.xml.template:102 Good idea, but not sure where should this check be made? In metastore client, metastore::main(), HMSHandler init, HiveConf::init() ? data/conf/hive-site.xml:71 Same as above. not sure where should this check be made? In metastore client, metastore::main(), HMSHandler init, HiveConf::init() ? REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244916#comment-13244916 ] Phabricator commented on HIVE-2585: --- cwsteinbach has requested changes to the revision "HIVE-2585 [jira] Collapse hive.metastore.uris and hive.metastore.local". INLINE COMMENTS conf/hive-default.xml.template:102 We should throw an error at startup if both hive.metastore.uris and javax.jdo.option.ConnectionURL are set. Giving silent preference to one of these properties is going to cause a lot of confusion for people who are using hive.metastore.local to quickly swap between local and remote metastores. data/conf/hive-site.xml:71 It would be nice to add a deprecation warning at startup if this is set. REVISION DETAIL https://reviews.facebook.net/D2559 BRANCH svn > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2585) Collapse hive.metastore.uris and hive.metastore.local
[ https://issues.apache.org/jira/browse/HIVE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244895#comment-13244895 ] Ashutosh Chauhan commented on HIVE-2585: {code} BUILD SUCCESSFUL Total time: 339 minutes 51 seconds {code} All the tests passed with this patch. > Collapse hive.metastore.uris and hive.metastore.local > - > > Key: HIVE-2585 > URL: https://issues.apache.org/jira/browse/HIVE-2585 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-2585.D2559.1.patch > > > We should just have hive.metastore.uris. If it is empty, we shall assume > local mode, if non-empty we shall use that string to connect to remote > metastore. Having two different keys for same information is confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira