virajjasani commented on code in PR #5718:
URL: https://github.com/apache/hadoop/pull/5718#discussion_r1226851855
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java:
##########
@@ -333,37 +335,31 @@ protected Path getCacheFilePath(final Configuration conf,
@Override
public void close() throws IOException {
- if (closed) {
- return;
- }
-
- closed = true;
+ if (closed.compareAndSet(false, true)) {
+ LOG.debug(getStats());
+ int numFilesDeleted = 0;
- LOG.info(getStats());
- int numFilesDeleted = 0;
-
- for (Entry entry : blocks.values()) {
- boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE,
PREFETCH_WRITE_LOCK_TIMEOUT,
- PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
- if (!lockAcquired) {
- LOG.error("Cache file {} deletion would not be attempted as write lock
could not"
- + " be acquired within {} {}", entry.path,
PREFETCH_WRITE_LOCK_TIMEOUT,
+ for (Entry entry : blocks.values()) {
+ boolean lockAcquired = entry.takeLock(Entry.LockType.WRITE,
PREFETCH_WRITE_LOCK_TIMEOUT,
PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
- continue;
- }
- try {
- Files.deleteIfExists(entry.path);
- prefetchingStatistics.blockRemovedFromFileCache();
- numFilesDeleted++;
- } catch (IOException e) {
- LOG.debug("Failed to delete cache file {}", entry.path, e);
- } finally {
- entry.releaseLock(Entry.LockType.WRITE);
+ if (!lockAcquired) {
+ LOG.error("Cache file {} deletion would not be attempted as write
lock could not"
+ + " be acquired within {} {}", entry.path,
PREFETCH_WRITE_LOCK_TIMEOUT,
+ PREFETCH_WRITE_LOCK_TIMEOUT_UNIT);
+ continue;
+ }
+ try {
+ Files.deleteIfExists(entry.path);
+ prefetchingStatistics.blockRemovedFromFileCache();
+ numFilesDeleted++;
+ } catch (IOException e) {
+ LOG.error("Failed to delete cache file {}", entry.path, e);
Review Comment:
sure let me make it at least warn, this is likely going to be rare event and
highlighting it would be helpful debugging stale file records in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
