Aman Sinha has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file ...................................................................... IMPALA-10934 (Part 1): Enable table definition over a single file Implements an abstraction layer to show files in a single directory. Impala side part - filesystem drivers are in HIVE-25569. Suppose that the filesystem has a directory in which there are multiple files: hdfs://somedir/f1.txt hdfs://somedir/f2.txt In case of a HMS backed table(s) - the contents of a directory could be considered as table. This patch enables a new file system wrapper 'sfs+' (sfs = single file system) which provides a view of a single file in a directory.' The '+' indicates that this wrapper can be added on top of multiple underlying file systems/object storage such as HDFS, S3 etc. The directory which contains the file could be specified: sfs+hdfs://somedir/f1.txt/#SINGLEFILE# This will be a directory containing only the f1.txt and nothing else. This patch was tested locally - with a custom build of Hive version which also had HIVE-25569. Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7 Reviewed-on: http://gerrit.cloudera.org:8080/17878 Reviewed-by: Aman Sinha <amsi...@cloudera.com> Tested-by: Aman Sinha <amsi...@cloudera.com> --- M be/src/exec/hdfs-table-sink.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/disk-io-mgr.cc M be/src/runtime/io/disk-io-mgr.h M be/src/util/hdfs-util.cc M be/src/util/hdfs-util.h M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M java/shaded-deps/hive-exec/pom.xml 8 files changed, 32 insertions(+), 2 deletions(-) Approvals: Aman Sinha: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7 Gerrit-Change-Number: 17878 Gerrit-PatchSet: 11 Gerrit-Owner: Anonymous Coward <k...@rxd.hu> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Anonymous Coward <k...@rxd.hu> Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
