Apache9 commented on code in PR #4541:
URL: https://github.com/apache/hbase/pull/4541#discussion_r899197367
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:
##########
@@ -314,20 +316,24 @@ public static void cleanExpiredMobFiles(FileSystem fs,
Configuration conf, Table
}
filesToClean
.add(new HStoreFile(fs, file.getPath(), conf, cacheConfig,
BloomType.NONE, true));
+ if (
+ filesToClean.size() >
conf.getInt(MOB_CLEANER_BATCH_SIZE_UPPER_BOUND,
+ DEFAULT_MOB_CLEANER_BATCH_SIZE_UPPER_BOUND)
+ ) {
+ try {
+ removeMobFiles(conf, fs, tableName, mobTableDir,
columnDescriptor.getName(),
+ filesToClean);
+ LOG.info("Table {} {} expired mob files are deleted", tableName,
filesToClean.size());
+ } catch (IOException e) {
+ LOG.error("Failed to delete the mob files, table {}", tableName,
e);
+ }
+ filesToClean.clear();
+ }
}
} catch (Exception e) {
LOG.error("Cannot parse the fileName " + fileName, e);
}
}
- if (!filesToClean.isEmpty()) {
- try {
- removeMobFiles(conf, fs, tableName, mobTableDir,
columnDescriptor.getName(), filesToClean);
- deletedFileCount = filesToClean.size();
- } catch (IOException e) {
- LOG.error("Failed to delete the mob files " + filesToClean, e);
- }
- }
- LOG.info("{} expired mob files are deleted", deletedFileCount);
Review Comment:
Let's still count the deleted files count and output it in the last? We can
also output everytime when we delete a batch.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:
##########
@@ -314,20 +316,24 @@ public static void cleanExpiredMobFiles(FileSystem fs,
Configuration conf, Table
}
filesToClean
.add(new HStoreFile(fs, file.getPath(), conf, cacheConfig,
BloomType.NONE, true));
+ if (
+ filesToClean.size() >
conf.getInt(MOB_CLEANER_BATCH_SIZE_UPPER_BOUND,
+ DEFAULT_MOB_CLEANER_BATCH_SIZE_UPPER_BOUND)
+ ) {
+ try {
+ removeMobFiles(conf, fs, tableName, mobTableDir,
columnDescriptor.getName(),
+ filesToClean);
+ LOG.info("Table {} {} expired mob files are deleted", tableName,
filesToClean.size());
+ } catch (IOException e) {
+ LOG.error("Failed to delete the mob files, table {}", tableName,
e);
+ }
+ filesToClean.clear();
+ }
}
} catch (Exception e) {
LOG.error("Cannot parse the fileName " + fileName, e);
}
}
- if (!filesToClean.isEmpty()) {
Review Comment:
We still need this piece of code? For the last round, where we do not have
10000 files, we will exit the loop with an unempty filesToClean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
