[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996224#comment-15996224 ]
Na Li edited comment on SENTRY-1649 at 5/5/17 8:29 PM: ------------------------------------------------------- The changes are 1. move starting HMSFollower from SentryService constructor to runServer() when starting SentryService. This allows getting notification after start()/stop() SentryService 2. create HMSFollower in runServer(). This allows HMSFollower to pick up different hive configuration after stop()/start() of SentryService. 3. HMSFollower.run() checks if the meta store URL is set in configuration. If not, return. This avoid creating local meta store. 4. Change HMSFollower initial start delay to 0 ms. Before, it was 60K ms (1 minutes). This zero delay allows test verification happens after HMSFollower gets permission from hive. 5. Close connections when HMSFollower.close() is called, which is called in SentryService.Stop(). 6. Keep setting if we should get full snapshot in HMSFollower constructor. Otherwise, when the first full snapshot has nothing, a default AuthzObj is created in sentrystore. Since the latest notification ID is still 0, next run tries to get full snapshot again. When the second full snapshot has nothing, we have "SentryAlreadyExistsException: AuthzObj: default already exists". So I move the code for setting needHiveSnapshot back to constructor. "SENTRY-1751 HMSFollower should not persist empty full snapshot" is created for this issue. 7. disable the unit test due to "SENTRY-1750 HMSFollower does not handle view update correctly". When TestHDFSIntegrationEnd2End.testViews() runs before TestHDFSIntegrationEnd2End.testColumnPrivileges(), the create view notification event causes exception in processing, and HMSFollower does not process any following event. That causes testColumnPrivileges() to fail. 8. Increase the time to wait from 100 ms to to twice of the HMSFollower task interval in testColumnPrivileges and related tests. In this way, HMSFollower should get the updates before verification. So the test is reliable. 9. fixed the failed test TestDbSentryOnFailureHookLoading. was (Author: linaataustin): The changes are 1. move starting HMSFollower from SentryService constructor to runServer() when starting SentryService. This allows getting notification after start()/stop() SentryService 2. create HMSFollower in runServer(). This allows HMSFollower to pick up different hive configuration after stop()/start() of SentryService. 3. HMSFollower.run() checks if the meta store URL is set in configuration. If not, return. This avoid creating local meta store. 4. Change HMSFollower initial start delay to 0 ms. Before, it was 60K ms (1 minutes). This zero delay allows test verification happens after HMSFollower gets permission from hive. 5. Close connections when HMSFollower.close() is called, which is called in SentryService.Stop(). 6. Keep setting if we should get full snapshot in HMSFollower constructor. Otherwise, when the first full snapshot has nothing, a default AuthzObj is created in sentrystore. Since the latest notification ID is still 0, next run tries to get full snapshot again. When the second full snapshot has nothing, we have "SentryAlreadyExistsException: AuthzObj: default already exists". So I move the code for setting needHiveSnapshot back to constructor. "SENTRY-1751 HMSFollower should not persist empty full snapshot" is created for this issue. 7. disable the unit test due to "SENTRY-1750 HMSFollower does not handle view update correctly". When TestHDFSIntegrationEnd2End.testViews() runs before TestHDFSIntegrationEnd2End.testColumnPrivileges(), the create view notification event causes exception in processing, and HMSFollower does not process any following event. That causes testColumnPrivileges() to fail. 8. Increase the time to wait from 100 ms to to twice of the HMSFollower task interval in testColumnPrivileges and related tests. In this way, HMSFollower should get the updates before verification. So the test is reliable. > Initialize HMSFollower when sentry server actually starts > --------------------------------------------------------- > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin > Affects Versions: sentry-ha-redesign > Reporter: Hao Hao > Assignee: Na Li > Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.015-sentry-ha-redesign.patch, > SENTRY-1649.022-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.025-sentry-ha-redesign.patch, > SENTRY-1649.028-sentry-ha-redesign.patch, > SENTRY-1649.029-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch, > SENTRY-1649.032-sentry-ha-redesign.patch, > SENTRY-1649.033-sentry-ha-redesign.patch, > SENTRY-1649.034-sentry-ha-redesign.patch, > SENTRY-1649.035-sentry-ha-redesign.patch, > SENTRY-1649.035-sentry-ha-redesign.patch, > SENTRY-1649.036-sentry-ha-redesign.patch, > SENTRY-1649.037-sentry-ha-redesign.patch, > SENTRY-1649.037-sentry-ha-redesign.patch, > SENTRY-1649.039-sentry-ha-redesign.patch, > SENTRY-1649.040-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)