Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17478 )
Change subject: IMPALA-10709: Min/max filters should be enabled for joins on sorted columns in Parquet tables ...................................................................... Patch Set 30: (4 comments) Thanks for working on this. I will review it tomorrow, I just had some early questions/comments. http://gerrit.cloudera.org:8080/#/c/17478/30//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17478/30//COMMIT_MSG@13 PS30, Line 13: only the leading column will be assigned a min/max filter In case of ZORDER I think we could use the other columns as well. Fast code path can still work to find the right pages if the BoundaryOrder happens to be ASC/DESC. http://gerrit.cloudera.org:8080/#/c/17478/30//COMMIT_MSG@26 PS30, Line 26: When minmax_filter_sorted_columns is set to false, no min/max filters : will be specifically assigned to the leading sort by columns. In that case the min/max filters are not used at all, right? http://gerrit.cloudera.org:8080/#/c/17478/30//COMMIT_MSG@35 PS30, Line 35: minmax_filter_fast_code_path If we have good confidence in the new algorithm then I think we can get rid of this query option. Maybe keep the VERIFICATION in debug builds. http://gerrit.cloudera.org:8080/#/c/17478/30/be/src/exec/parquet/hdfs-parquet-scanner.cc File be/src/exec/parquet/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/17478/30/be/src/exec/parquet/hdfs-parquet-scanner.cc@1196 PS30, Line 1196: ASCENDING Do we plan to make it work for DESCENDING as well? Though I'm not sure if it would be really useful, since probably an ordered write will always mean ASCENDING order. -- To view, visit http://gerrit.cloudera.org:8080/17478 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28c19c4b39b01ffa7d275fb245be85c28e9b2963 Gerrit-Change-Number: 17478 Gerrit-PatchSet: 30 Gerrit-Owner: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Thu, 03 Jun 2021 17:27:06 +0000 Gerrit-HasComments: Yes
