[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731674#comment-17731674 ]
ASF GitHub Bot commented on HADOOP-18756: ----------------------------------------- virajjasani commented on code in PR #5718: URL: https://github.com/apache/hadoop/pull/5718#discussion_r1226902160 ########## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java: ########## @@ -333,37 +335,31 @@ protected Path getCacheFilePath(final Configuration conf, @Override public void close() throws IOException { - if (closed) { - return; - } - - closed = true; + if (closed.compareAndSet(false, true)) { + LOG.debug(getStats()); + int numFilesDeleted = 0; - LOG.info(getStats()); - int numFilesDeleted = 0; - - for (Entry entry : blocks.values()) { - boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE, PREFETCH_WRITE_LOCK_TIMEOUT, - PREFETCH_WRITE_LOCK_TIMEOUT_UNIT); - if (!lockAcquired) { - LOG.error("Cache file {} deletion would not be attempted as write lock could not" - + " be acquired within {} {}", entry.path, PREFETCH_WRITE_LOCK_TIMEOUT, + for (Entry entry : blocks.values()) { + boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE, PREFETCH_WRITE_LOCK_TIMEOUT, PREFETCH_WRITE_LOCK_TIMEOUT_UNIT); - continue; - } - try { - Files.deleteIfExists(entry.path); - prefetchingStatistics.blockRemovedFromFileCache(); - numFilesDeleted++; - } catch (IOException e) { - LOG.debug("Failed to delete cache file {}", entry.path, e); - } finally { - entry.releaseLock(Entry.LockType.WRITE); + if (!lockAcquired) { + LOG.error("Cache file {} deletion would not be attempted as write lock could not" + + " be acquired within {} {}", entry.path, PREFETCH_WRITE_LOCK_TIMEOUT, + PREFETCH_WRITE_LOCK_TIMEOUT_UNIT); + continue; + } + try { + Files.deleteIfExists(entry.path); + prefetchingStatistics.blockRemovedFromFileCache(); + numFilesDeleted++; + } catch (IOException e) { + LOG.error("Failed to delete cache file {}", entry.path, e); Review Comment: with this new addendum (log level change: ERROR to WARN), i re-ran the test suite again (without scale profile, since the addendum change is trivial) > CachingBlockManager to use AtomicBoolean for closed flag > -------------------------------------------------------- > > Key: HADOOP-18756 > URL: https://issues.apache.org/jira/browse/HADOOP-18756 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.3.9 > Reporter: Steve Loughran > Assignee: Viraj Jasani > Priority: Major > Labels: pull-request-available > > the {{CachingBlockManager}} uses the boolean field {{closed)) in various > operations, including a do/while loop. to ensure the flag is correctly > updated across threads, it needs to move to an atomic boolean. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org