> On Sept. 28, 2018, 4:56 p.m., Sergio Pena wrote: > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java > > Line 158 (original), 168-172 (patched) > > <https://reviews.apache.org/r/68547/diff/6/?file=2092470#file2092470line168> > > > > I don't think this logic whether use a cache or not should be in this > > clas. The retriever should call the cache instead (or call the SentryStore > > if it needs to invalidate the current cache image). > > Arjun Mishra wrote: > This is because the logic to invalidate cache is dependent on whether > delta updates are being sent or not. Since the decisision to send delta > updates is done in DBUpdateForwarder this logic sits in this piece of code > > Sergio Pena wrote: > Do you need the delta number to invalidate metadata for a snapshot? The > PathImageRetriever can keep a cache of the current snapshot and invalidate > its cache if the next snapshot ID is different, otherwise return the full > path image from the cache (read a SoftReference example on how help clean the > cache if it is not used).
Yes we need delta numbers. If the image id is the same but the sequence number requested is greater than cache sequence number by 1 we need to request a a full update for sequence numbers that don't exist in sentry db - Arjun ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68547/#review209098 ----------------------------------------------------------- On Sept. 28, 2018, 7:56 p.m., Arjun Mishra wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68547/ > ----------------------------------------------------------- > > (Updated Sept. 28, 2018, 7:56 p.m.) > > > Review request for sentry, kalyan kumar kalvagadda, Na Li, and Sergio Pena. > > > Bugs: SENTRY-2370 > https://issues.apache.org/jira/browse/SENTRY-2370 > > > Repository: sentry > > > Description > ------- > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN > > > Diffs > ----- > > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java > 08b16a4df > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathsUpdateImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermissionsUpdateImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java > b8f5ce7db > > sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestDBUpdateForwarder.java > f86ce6f83 > > > Diff: https://reviews.apache.org/r/68547/diff/7/ > > > Testing > ------- > > > Thanks, > > Arjun Mishra > >
