-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58975/
-----------------------------------------------------------
(Updated May 18, 2017, 3:08 p.m.)
Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena,
Vamsee Yarlagadda, and Vadim Spector.
Changes
-------
Fixed issues observed in tests.
HMS has a default database with name "default".
Behavior in old patch:
1. When HMS runs for the first time, it takes a snapshot which just has one
entry for database "default".
2. lastProcessedNotificationID would be 0 as there were no notifications.
3. For couple of seconds, Every time HMSFollower runs, it initializes
lastProcessedNotificationID with 0. As the changes done were dependent on
lastProcessedNotificationID to decide if a snapshot is needed, HMSFollower used
to request snapshot every time it ran.
4. After some time when the notifications were inserted into notification log
table of HMS, HMSFollower never received notifications as there out-of-sequence
exceptions raised for the requests made by HMSFollower.
Behavior in New patch:
1. When HMS runs for the first time, it takes a snapshot which just has one
entry for database "default".
2. lastProcessedNotificationID would be 0 as there were no notifications.
3. For couple of seconds, Every time HMSFollower runs, it initializes
lastProcessedNotificationID with 0. Instead of depending on
lastProcessedNotificationID retrieved from database to decide if snapshot is
needed, HMSFollower depends on it's static data. That way HMSFollower, does a
snapshot only once. This was the older behavior. Just reverting back to older
behavior.
This change solved the issue.
There is some inherent issue(out-of-sequence) with the communication with HMS
which I will debug further and raise a bug either on SENTRY/HIVE based in my
finding. As far as this jira is concerned, HMSFollower should anyway request a
snapshot only once this new patch should be good.
Bugs: SENTRY-1669
https://issues.apache.org/jira/browse/SENTRY-1669
Repository: sentry
Description
-------
Sentry should store Id's of all the notifications that it processed in a
seperate database.
HMSFollower before every run should get the last notification ID from this
persistent store.
Diffs (updated)
-----
b/sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryHmsNotification.java
PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPathChange.java
42f80aa
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
8d9528f
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
8fd5278
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
709d195
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
c458651
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
5dc0ad3
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
083e0ac
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
e7443eb
Diff: https://reviews.apache.org/r/58975/diff/15/
Changes: https://reviews.apache.org/r/58975/diff/14-15/
Testing
-------
Sentry stores Id's of all the notifications that it processed in a seperate
database.
HMSFollower before every run gets the last notification ID from this persistent
store.
Thanks,
kalyan kumar kalvagadda
