> On Sept. 28, 2018, 4:56 p.m., Sergio Pena wrote: > > What if NN does not request full images anymore? How can the cache be > > released from memory to avoid unused cache for the rest of the process life? > > Arjun Mishra wrote: > Sergio, the cache is releases as soon as Deltas are being sent to NN > > Sergio Pena wrote: > what if there are no Deltas available to send in the next NN requests? > The cache will live on memory until then. > Look for Java soft or weak references (and the use of SoftReferences or > WeakReferences). These objects might help on letting the GC to clean your > cache if it is not needed anymore.
Deltas don't have to be sent they only have to be requested by NN. As soon as NN gets a full update, it will start requesting delta updates and just the trigger for requesting delta updates will invalidate cache I will research soft references - Arjun ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68547/#review209098 ----------------------------------------------------------- On Sept. 28, 2018, 7:56 p.m., Arjun Mishra wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68547/ > ----------------------------------------------------------- > > (Updated Sept. 28, 2018, 7:56 p.m.) > > > Review request for sentry, kalyan kumar kalvagadda, Na Li, and Sergio Pena. > > > Bugs: SENTRY-2370 > https://issues.apache.org/jira/browse/SENTRY-2370 > > > Repository: sentry > > > Description > ------- > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN > > > Diffs > ----- > > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java > 08b16a4df > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathsUpdateImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermissionsUpdateImageCache.java > PRE-CREATION > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java > b8f5ce7db > > sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestDBUpdateForwarder.java > f86ce6f83 > > > Diff: https://reviews.apache.org/r/68547/diff/7/ > > > Testing > ------- > > > Thanks, > > Arjun Mishra > >
