xiangfu0 commented on code in PR #18457:
URL: https://github.com/apache/pinot/pull/18457#discussion_r3218773071
##########
pinot-plugins/pinot-file-system/pinot-hdfs/src/main/java/org/apache/pinot/plugin/filesystem/HadoopPinotFS.java:
##########
@@ -183,6 +183,9 @@ private void visitFileStatus(Path path, boolean recursive,
Consumer<FileStatus>
throws IOException {
// _hadoopFS.listFiles(path, false) will not return directories as files,
thus use listStatus(path) here.
FileStatus[] files = _hadoopFS.listStatus(path);
+ if (files == null) {
Review Comment:
Older Hadoop implementations such as `RawLocalFileSystem` can return `null`
here when the directory listing fails. Treating that as an empty directory
changes `delete(uri, false)` from fail-fast into `_hadoopFS.delete(path,
true)`, which can recursively delete a non-empty directory that should have
been preserved. Please keep this path throwing rather than silently returning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
