Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14711 )
Change subject: IMPALA-8778: Support Apache Hudi Read Optimized Table ...................................................................... Patch Set 16: (2 comments) http://gerrit.cloudera.org:8080/#/c/14711/16/be/src/exec/hdfs-scan-node-base.cc File be/src/exec/hdfs-scan-node-base.cc: http://gerrit.cloudera.org:8080/#/c/14711/16/be/src/exec/hdfs-scan-node-base.cc@379 PS16, Line 379: HUDI_PARQUET > I see, but in the backend you just create "low-level" operators, such as sc Agree with Zoltan. It would be nice if there were no necessary backend changes here. While this might change in the future (e.g. when Hudi near real-time format is added), I think we can worry about that in a separate patch. I'm actually surprised this doesn't already work with your current patch. Reading the code Zoltan linked (https://gerrit.cloudera.org/#/c/14711/7/fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java line 180) that serializes all THdfsFileFormat#HUDI_PARQUET as THdfsFileFormat#PARQUET. That being said, I can't point to the place in the planner where you would need to make this change (maybe someone more familiar with the planner code would know). So this isn't a blocker for me. http://gerrit.cloudera.org:8080/#/c/14711/16/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: http://gerrit.cloudera.org:8080/#/c/14711/16/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@354 PS16, Line 354: isParquet > maybe 'isParquetBased'? +1 -- To view, visit http://gerrit.cloudera.org:8080/14711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65e146b347714df32fe968409ef2dde1f6a25cdf Gerrit-Change-Number: 14711 Gerrit-PatchSet: 16 Gerrit-Owner: Yanjia Gary Li <yanjia.gary...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Norbert Luksa <norbert.lu...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Yanjia Gary Li <yanjia.gary...@gmail.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 05 Feb 2020 16:35:17 +0000 Gerrit-HasComments: Yes
