[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548140#comment-13548140 ] Hudson commented on HIVE-2716: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-2833 Fix test failures caused by HIVE-2716 (Kevin Wilfong via namit) (Revision 1296946) HIVE-2716 : Move retry logic in HiveMetaStore to a separe class (enis via hashutosh) (Revision 1292420) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1296946 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyJdoConnectionUrlHook.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreConnectionUrlHook.java hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1292420 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.9.0 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2716.D1227.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2716.D1227.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2716.D1227.3.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2716.D1227.4.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2716.D2055.1.patch, HIVE-2716.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13222306#comment-13222306 ] Hudson commented on HIVE-2716: -- Integrated in Hive-trunk-h0.21 #1290 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1290/]) HIVE-2833 Fix test failures caused by HIVE-2716 (Kevin Wilfong via namit) (Revision 1296946) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1296946 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyJdoConnectionUrlHook.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreConnectionUrlHook.java Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.9.0 Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, HIVE-2716.D1227.3.patch, HIVE-2716.D1227.4.patch, HIVE-2716.D2055.1.patch, HIVE-2716.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221324#comment-13221324 ] Kevin Wilfong commented on HIVE-2716: - Ignore the Phabricator comment and patch for D2055, that was intended for another JIRA. Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.9.0 Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, HIVE-2716.D1227.3.patch, HIVE-2716.D1227.4.patch, HIVE-2716.D2055.1.patch, HIVE-2716.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214241#comment-13214241 ] Ashutosh Chauhan commented on HIVE-2716: I reran the ant test -Dtestcase=TestNegativeCliDriver -Dqfile=script_broken_pipe1.q which failed as reported and it passed. So, looks like that failure is not related to this patch. Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.9.0 Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, HIVE-2716.D1227.3.patch, HIVE-2716.D1227.4.patch, HIVE-2716.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13212254#comment-13212254 ] Phabricator commented on HIVE-2716: --- ashutoshc has accepted the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. +1 will commit if tests pass. REVISION DETAIL https://reviews.facebook.net/D1227 BRANCH ms-retry-logic Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, HIVE-2716.D1227.3.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13212385#comment-13212385 ] Ashutosh Chauhan commented on HIVE-2716: ant test -Dtestcase=TestNegativeCliDriver -Dqfile=authorization_fail_1.q failed. Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, HIVE-2716.D1227.3.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209958#comment-13209958 ] Phabricator commented on HIVE-2716: --- cwsteinbach has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. What's the motivation for this patch? Is this change required by something on the HCat side? REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209965#comment-13209965 ] Phabricator commented on HIVE-2716: --- ashutoshc has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. Enis will know more, but AFAIK HCatalog won't get anything out of this change. I like the idea of the patch for refactoring HiveMetaStore.java which is unwieldy in its current form, thus making Hive codebase easier to read and follow. Do you have any specific concerns? REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209975#comment-13209975 ] Phabricator commented on HIVE-2716: --- enis has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. What's the motivation for this patch? Is this change required by something on the HCat side? I created the issue and the patch while working on the HMS. The patch refactors the retry logic out of HMS. The Command wrapper inside HMS makes the code very hard to follow, and requires some ugly code for handling checked exceptions, and is error prone is someone forgets wrapping a new method in RawStore. REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209984#comment-13209984 ] Phabricator commented on HIVE-2716: --- cwsteinbach has requested changes to the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. INLINE COMMENTS metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:136 Is this change required, or are you just providing coverage for the new dropType(String, boolean) method? Were exceptions somehow getting masked earlier? metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java:38 It's not clear to me why HCatalog needs access to this class. Can you please provide more details? REVISION DETAIL https://reviews.facebook.net/D1227 BRANCH ms-retry-logic Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202683#comment-13202683 ] Ashutosh Chauhan commented on HIVE-2716: @Enis, Patch has gone stale. Can you rebase it ? Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185284#comment-13185284 ] Phabricator commented on HIVE-2716: --- enis has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. Some context for the patch: - All the retries in HiveMetaStore are actually retying methods of the RawStore interface, which handles the jdbc connection. Thus an abstraction in this layer makes more sense, and removes the clutter in the calling code. - The RawStore is proxied to introduce an interceptor to carry out the method invocations. Proxies of this type are used by most of the ORM frameworks for transaction management, or retrying, logging, etc. Performance-wise, there is a lot of layers already, like the thrift call - jdo - jdbc - SQL parse - db engine that it won't make a major difference to use a dynamic proxy. Besides they are not very slow as believed. - HiveMetaStore.Command and executeWithRetry() is limited-provate to HCat, and Hcat does not use these interfaces, so I just removed them. If we need them, we can instead deprecate. REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
