[
https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731673#comment-17731673
]
ASF GitHub Bot commented on HADOOP-18756:
-----------------------------------------
virajjasani commented on code in PR #5718:
URL: https://github.com/apache/hadoop/pull/5718#discussion_r1226902160
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java:
##########
@@ -333,37 +335,31 @@ protected Path getCacheFilePath(final Configuration conf,
@Override
public void close() throws IOException {
- if (closed) {
- return;
- }
-
- closed = true;
+ if (closed.compareAndSet(false, true)) {
+ LOG.debug(getStats());
+ int numFilesDeleted = 0;
- LOG.info(getStats());
- int numFilesDeleted = 0;
-
- for (Entry entry : blocks.values()) {
- boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE,
PREFETCH_WRITE_LOCK_TIMEOUT,
- PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
- if (!lockAcquired) {
- LOG.error("Cache file {} deletion would not be attempted as write lock
could not"
- + " be acquired within {} {}", entry.path,
PREFETCH_WRITE_LOCK_TIMEOUT,
+ for (Entry entry : blocks.values()) {
+ boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE,
PREFETCH_WRITE_LOCK_TIMEOUT,
PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
- continue;
- }
- try {
- Files.deleteIfExists(entry.path);
- prefetchingStatistics.blockRemovedFromFileCache();
- numFilesDeleted++;
- } catch (IOException e) {
- LOG.debug("Failed to delete cache file {}", entry.path, e);
- } finally {
- entry.releaseLock(Entry.LockType.WRITE);
+ if (!lockAcquired) {
+ LOG.error("Cache file {} deletion would not be attempted as write
lock could not"
+ + " be acquired within {} {}", entry.path,
PREFETCH_WRITE_LOCK_TIMEOUT,
+ PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
+ continue;
+ }
+ try {
+ Files.deleteIfExists(entry.path);
+ prefetchingStatistics.blockRemovedFromFileCache();
+ numFilesDeleted++;
+ } catch (IOException e) {
+ LOG.error("Failed to delete cache file {}", entry.path, e);
Review Comment:
with this new addendum (log level change: ERROR to WARN), i re-ran the test
suite again (without scale profile)
> CachingBlockManager to use AtomicBoolean for closed flag
> --------------------------------------------------------
>
> Key: HADOOP-18756
> URL: https://issues.apache.org/jira/browse/HADOOP-18756
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
>
> the {{CachingBlockManager}} uses the boolean field {{closed)) in various
> operations, including a do/while loop. to ensure the flag is correctly
> updated across threads, it needs to move to an atomic boolean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
