elek commented on issue #726: HDDS-3267. replace ContainerCache in BlockUtils by LoadingCache URL: https://github.com/apache/hadoop-ozone/pull/726#issuecomment-613996359 Thanks the @esahekmat the update. I am not convinced if the the proposed patch (using `weakValues()`) solves the problem. It's about the GC, not about the eviction AFAIK. Setting the ` .maximumSize()` using `OZONE_CONTAINER_CACHE_SIZE` is also missing from the new code. Let's assume that the `maximumSize` is set to `2`: ```java cache = CacheBuilder.newBuilder() .weakValues() .maximumSize(2) .removalListener((notification) -> ((ReferenceDB) notification.getValue()).cleanup()) .concurrencyLevel(1) .... ``` In this case the DB reference can be evicted even if an obtained reference is used: ```java OzoneConfiguration conf = new OzoneConfiguration(); conf.setInt(OzoneConfigKeys.OZONE_CONTAINER_CACHE_SIZE, 2); KeyValueContainerData container1instance1 = createContainer(1L); KeyValueContainerData container1instance2 = createContainer(1L); ReferenceDB dba = BlockUtils.getDB(container1instance1, conf); ReferenceDB dbb = BlockUtils.getDB(container1instance2, conf); //should be the same, if cache is used Assert.assertEquals(dba, dbb); //closing first reference, second is still used dba.close(); //works well as the db itself is not closed, yet dbb.getStore().get("asd".getBytes()); //force eviction with adding two more containers. BlockUtils.getDB(createContainer(3L), conf); BlockUtils.getDB(createContainer(4L), conf); //doesn't work, even if the dbb is not yet closed. dbb.getStore().get("asd".getBytes()); ``` The last call throws a `NullPointerException`. The existing code (based on my understanding) works well, because it closes the DB only it there are no more ("unclosed") references to the DB.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
