[
https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996224#comment-15996224
]
Na Li edited comment on SENTRY-1649 at 5/5/17 8:27 PM:
-------------------------------------------------------
The changes are
1. move starting HMSFollower from SentryService constructor to runServer() when
starting SentryService. This allows getting notification after start()/stop()
SentryService
2. create HMSFollower in runServer(). This allows HMSFollower to pick up
different hive configuration after stop()/start() of SentryService.
3. HMSFollower.run() checks if the meta store URL is set in configuration. If
not, return. This avoid creating local meta store.
4. Change HMSFollower initial start delay to 0 ms. Before, it was 60K ms (1
minutes). This zero delay allows test verification happens after HMSFollower
gets permission from hive.
5. Close connections when HMSFollower.close() is called, which is called in
SentryService.Stop().
6. Keep setting if we should get full snapshot in HMSFollower constructor.
Otherwise, when the first full snapshot has nothing, a default AuthzObj is
created in sentrystore. Since the latest notification ID is still 0, next run
tries to get full snapshot again. When the second full snapshot has nothing, we
have "SentryAlreadyExistsException: AuthzObj: default already exists". So I
move the code for setting needHiveSnapshot back to constructor. "SENTRY-1751
HMSFollower should not persist empty full snapshot" is created for this issue.
7. disable the unit test due to "SENTRY-1750 HMSFollower does not handle view
update correctly". When TestHDFSIntegrationEnd2End.testViews() runs before
TestHDFSIntegrationEnd2End.testColumnPrivileges(), the create view notification
event causes exception in processing, and HMSFollower does not process any
following event. That causes testColumnPrivileges() to fail.
8. Increase the time to wait from 100 ms to to twice of the HMSFollower task
interval in testColumnPrivileges and related tests. In this way, HMSFollower
should get the updates before verification. So the test is reliable.
was (Author: linaataustin):
The changes are
1. move starting HMSFollower from SentryService constructor to runServer() when
starting SentryService. This allows getting notification after start()/stop()
SentryService
2. create HMSFollower in runServer(). This allows HMSFollower to pick up
different hive configuration after stop()/start() of SentryService.
3. HMSFollower.run() checks if the meta store URL is set in configuration. If
not, return. This avoid creating local meta store.
4. Change HMSFollower initial start delay to 0 ms. Before, it was 60K ms (1
minutes). This zero delay allows test verification happens after HMSFollower
gets permission from hive.
5. Close connections when HMSFollower.close() is called, which is called in
SentryService.Stop().
6. Keep setting if we should get full snapshot in HMSFollower constructor.
Otherwise, when the first full snapshot has nothing, a default AuthzObj is
created in sentrystore. Since the latest notification ID is still 0, next run
tries to get full snapshot again. When the second full snapshot has nothing, we
have "SentryAlreadyExistsException: AuthzObj: default already exists". So I
move the code for setting needHiveSnapshot back to constructor.
> Initialize HMSFollower when sentry server actually starts
> ---------------------------------------------------------
>
> Key: SENTRY-1649
> URL: https://issues.apache.org/jira/browse/SENTRY-1649
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
> Affects Versions: sentry-ha-redesign
> Reporter: Hao Hao
> Assignee: Na Li
> Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1649.015-sentry-ha-redesign.patch,
> SENTRY-1649.022-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.025-sentry-ha-redesign.patch,
> SENTRY-1649.028-sentry-ha-redesign.patch,
> SENTRY-1649.029-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.031-sentry-ha-redesign.patch,
> SENTRY-1649.031-sentry-ha-redesign.patch,
> SENTRY-1649.031-sentry-ha-redesign.patch,
> SENTRY-1649.032-sentry-ha-redesign.patch,
> SENTRY-1649.033-sentry-ha-redesign.patch,
> SENTRY-1649.034-sentry-ha-redesign.patch,
> SENTRY-1649.035-sentry-ha-redesign.patch,
> SENTRY-1649.035-sentry-ha-redesign.patch,
> SENTRY-1649.036-sentry-ha-redesign.patch,
> SENTRY-1649.037-sentry-ha-redesign.patch,
> SENTRY-1649.037-sentry-ha-redesign.patch,
> SENTRY-1649.039-sentry-ha-redesign.patch,
> SENTRY-1649.040-sentry-ha-redesign.patch
>
>
> Now HMSFollower has been initialized at the constructor of SentryService. It
> would be better to initialize it when the service starts, e.g runServer().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
