[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730651#comment-17730651 ]
ASF GitHub Bot commented on HADOOP-18756: ----------------------------------------- steveloughran commented on code in PR #5718: URL: https://github.com/apache/hadoop/pull/5718#discussion_r1223339252 ########## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java: ########## @@ -333,37 +335,33 @@ protected Path getCacheFilePath(final Configuration conf, @Override public void close() throws IOException { - if (closed) { - return; - } - - closed = true; + if (closed.compareAndSet(false, true)) { + LOG.info(getStats()); Review Comment: could we log this at debug; prefetching is fairly noisy right now ########## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java: ########## @@ -258,27 +260,23 @@ protected Path getCacheFilePath(final Configuration conf, @Override public void close() throws IOException { - if (closed) { - return; - } - - closed = true; - - LOG.info(getStats()); - int numFilesDeleted = 0; - - for (Entry entry : blocks.values()) { - try { - Files.deleteIfExists(entry.path); - prefetchingStatistics.blockRemovedFromFileCache(); - numFilesDeleted++; - } catch (IOException e) { - LOG.debug("Failed to delete cache file {}", entry.path, e); + if (closed.compareAndSet(false, true)) { + LOG.info(getStats()); + int numFilesDeleted = 0; + + for (Entry entry : blocks.values()) { + try { + Files.deleteIfExists(entry.path); + prefetchingStatistics.blockRemovedFromFileCache(); + numFilesDeleted++; + } catch (IOException e) { + LOG.debug("Failed to delete cache file {}", entry.path, e); + } } - } - if (numFilesDeleted > 0) { - LOG.info("Deleted {} cache files", numFilesDeleted); + if (numFilesDeleted > 0) { + LOG.info("Deleted {} cache files", numFilesDeleted); Review Comment: downgrade to debug and remove the if() test; we can log the zero count too > CachingBlockManager to use AtomicBoolean for closed flag > -------------------------------------------------------- > > Key: HADOOP-18756 > URL: https://issues.apache.org/jira/browse/HADOOP-18756 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.3.9 > Reporter: Steve Loughran > Assignee: Viraj Jasani > Priority: Major > Labels: pull-request-available > > the {{CachingBlockManager}} uses the boolean field {{closed)) in various > operations, including a do/while loop. to ensure the flag is correctly > updated across threads, it needs to move to an atomic boolean. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org