Hello Quanlong Huang, Qifan Chen, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17316 to look at the new patch set (#2). Change subject: IMPALA-10658: LOAD DATA INPATH silently fails between HDFS and Azure ABFS ...................................................................... IMPALA-10658: LOAD DATA INPATH silently fails between HDFS and Azure ABFS LOAD DATA INPATH silently fails when Impala tries to move files from HDFS to ABFS. The problem is that we use FileSystem.makeQualified(Path) to decide if path is on a given filesystem. We expect to get an IllegalArgumentException if path is on a different filesystem. However, the Azure FileSystem implementation doesn't throw this exception. Because of that Impala thinks that an 'hdfs://' path and an 'abfs://' path is on the same filesystem, so it tries to move files with FileSystem.rename(). In case of errors rename() might throw an Exception, or return false. Impala doesn't check the return value, therefore if rename() returns false then the error remains silent. This patch fixes Impala's isPathOnFileSystem() and also adds a check for the return value of rename(). Testing: * tested manually between HDFS and Azure ABFS. * added JUnit test to FileSystemUtilTest Change-Id: Id807e8a200b83283a09d3a917185cabab930017d --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java 2 files changed, 53 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/17316/2 -- To view, visit http://gerrit.cloudera.org:8080/17316 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id807e8a200b83283a09d3a917185cabab930017d Gerrit-Change-Number: 17316 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>