ayushtkn commented on code in PR #3531:
URL: https://github.com/apache/hive/pull/3531#discussion_r952495175
##########
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:
##########
@@ -1499,10 +1500,10 @@ private static ValidTxnList
getValidTxnList(Configuration conf) {
public static Map<Path, HdfsDirSnapshot> getHdfsDirSnapshotsForCleaner(final
FileSystem fs, final Path path)
throws IOException {
Map<Path, HdfsDirSnapshot> dirToSnapshots = new HashMap<>();
- Deque<RemoteIterator<FileStatus>> stack = new ArrayDeque<>();
- stack.push(fs.listStatusIterator(path));
+ Deque<RemoteIterator<LocatedFileStatus>> stack = new ArrayDeque<>();
+ stack.push(FileUtils.listAndFilter (fs, path, true, acidHiddenFileFilter));
Review Comment:
Earlier it was listStatusIterator, you changed it to listFiles with
recursive, it won't be having directories now, empty directories and stuff like
that will get completely skipped, I am not sure if you can do that. Lets not
change the behaviour in this scope.
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FileUtils.java:
##########
@@ -654,4 +655,11 @@ void findNext() throws IOException {
nextFile = null;
}
}
+
+ public static RemoteIterator<LocatedFileStatus> listAndFilter(FileSystem
fileSystem, Path path, boolean recursive,
+ PathFilter
filter) throws IOException {
+ return RemoteIterators.filteringRemoteIterator(
+ fileSystem.listFiles(path, recursive),
+ status -> filter.accept(status.getPath()));
+ }
Review Comment:
Code isn't formatted. Use `eclipse-styles.xml` present in dev-support in
hive repo to format the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
