----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57232/#review169817 -----------------------------------------------------------
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java Lines 36 (patched) <https://reviews.apache.org/r/57232/#comment242455> This class and its methods can be package-private sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java Lines 49 (patched) <https://reviews.apache.org/r/57232/#comment242460> I had this comment before - it is cleaner to be explicit whether we return a list of deltas or a full update - either by using different functions or returning a special result. sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java Lines 67 (patched) <https://reviews.apache.org/r/57232/#comment242456> Use collections.emptyList() instead sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java Lines 72 (patched) <https://reviews.apache.org/r/57232/#comment242458> Add extra parenthesis for clarity: if ((seqNum != SentryStore.INIT_CHANGE_ID) && deltaRetriever.isDeltaAvailable(seqNum)) { sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java Lines 42 (patched) <https://reviews.apache.org/r/57232/#comment242473> Would it make sense to add a metric to measure time to get it? If you don't want to add it now, please don't forget to file a follow-up JIRA. Same for PermDeltaRetriever. sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java Lines 43 (patched) <https://reviews.apache.org/r/57232/#comment242467> You can return Collections.emptyList() if mSentryPathChange is empty. Otherwise it is better to specify exact size for the update: Collection<PathsUpdate> updates = new ArrayList<>(mSentryPathChanges.size()); Same comment for PermDeltaRetriever. sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java Lines 75 (patched) <https://reviews.apache.org/r/57232/#comment242475> Do we need to construct a new array each time? Can we have this as a static constant? At the minimum, use static string for "/". sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java Lines 77 (patched) <https://reviews.apache.org/r/57232/#comment242476> Since there is no concurrency here, why should we create a new lock each time? A single lock would work as well. sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java Line 79 (original), 79 (patched) <https://reviews.apache.org/r/57232/#comment242477> can we use Collections.emptyList() instead? sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java Line 49 (original), 49 (patched) <https://reviews.apache.org/r/57232/#comment242478> Are we still using SentryPlugin? sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3296 (patched) <https://reviews.apache.org/r/57232/#comment242481> Why use 't' as the parameter rather then changeId? sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3299 (patched) <https://reviews.apache.org/r/57232/#comment242482> simply return !changes.isEMpty() sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3380 (patched) <https://reviews.apache.org/r/57232/#comment242484> instead of t use a better name like changeId sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3403 (patched) <https://reviews.apache.org/r/57232/#comment242486> Add parenthesis: if (pathChanges.size() < ((curChangeID - changeID) + 1)) { sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3406 (patched) <https://reviews.apache.org/r/57232/#comment242487> return Collections.emptyList() sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3432 (patched) <https://reviews.apache.org/r/57232/#comment242488> Add parenthesis sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3435 (patched) <https://reviews.apache.org/r/57232/#comment242489> return Collections.emptyList() - Alexander Kolbasov On March 22, 2017, 11:08 p.m., Hao Hao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57232/ > ----------------------------------------------------------- > > (Updated March 22, 2017, 11:08 p.m.) > > > Review request for sentry. > > > Repository: sentry > > > Description > ------- > > Add the logic for propagating Sentry Permissions or Sentry representation of > HMS Paths, from a persistent storage instead of in memoery cache of Sentry > service, to a Sentry consumer such as HDFS NameNode. It includes: > 1. Delta update retrieval logic from persistent storage. > 2. Propagation logic for consumer to get the delta update or a complete > snapshot. > > > Diffs > ----- > > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java > 0e40756b0da0214e83b7127dcd8816f30c9692c8 > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java > 69aa098bc71362b1aba0ad219483cfce7d389964 > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java > 085971b34e3901b7a1d59bd8e7516b25f81ca872 > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java > 16a1604700079cb37f6fbaeceae92c67b2c08d8b > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java > 3017c9edb2217cd0152183f927691311a276e850 > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java > e4f3f580ee0a6bf83d5a32001bff130c0bff13aa > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java > be145695af35b34f409e1f8b6e49da247f78e7a0 > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java > 029f9d5dc89493f7e7086b80387f71fef1ac0805 > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java > 22c57690a0f74822187335f97cf1eede8d415acf > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java > 03c67d674813c59a23c5bc6c57671d79dfd235a0 > > sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java > d12b1342116217685a46367567e2c0f8d7c288e5 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java > 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java > f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 > > sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java > 75f855caec1060f9158562aeb81ec53d8a679941 > > sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java > 5e8b2fac76d6dcf7aee34729e86755797af64406 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java > 859c8f882db76e46297c1f6377de32bc2b08f485 > > > Diff: https://reviews.apache.org/r/57232/diff/7/ > > > Testing > ------- > > > Thanks, > > Hao Hao > >
