Gergely Fürnstáhl has uploaded a new patch set (#17). ( http://gerrit.cloudera.org:8080/19850 )
Change subject: WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator ...................................................................... WIP IMPALA-11619: Improve Iceberg V2 reads with a custom Iceberg Position Delete operator Implemented new exec node to optimize iceberg v2 read performance. IcebergDeleteNode and IcebergDeleteBuild classes are based on PartitionedHashJoin counterparts. The actual "join" part of the node is optimized, while others are kept very similarly, to be able to integrate features of PartitionedHashJoin if needed (partitioning, spilling). Still TODO: - iceberg delete node specific compute costs and broadcast/distribution mode deciding logic Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa --- M be/src/codegen/impala-ir.cc M be/src/exec/CMakeLists.txt M be/src/exec/blocking-join-node.cc M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.cc A be/src/exec/iceberg-delete-builder.cc A be/src/exec/iceberg-delete-builder.h A be/src/exec/iceberg-delete-node.cc A be/src/exec/iceberg-delete-node.h A be/src/exec/iceberg-delete-node.inline.h M be/src/exec/join-builder.h M be/src/exec/join-op.h M be/src/runtime/query-state.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/DataSinks.thrift M common/thrift/ImpalaService.thrift M common/thrift/PlanNodes.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/JoinOperator.java M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java A fe/src/main/java/org/apache/impala/planner/IcebergDeleteNode.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/main/java/org/apache/impala/planner/JoinNode.java M fe/src/main/java/org/apache/impala/planner/NestedLoopJoinNode.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M tests/query_test/test_iceberg.py 31 files changed, 1,899 insertions(+), 103 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/19850/17 -- To view, visit http://gerrit.cloudera.org:8080/19850 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I024a61573c83bda5584f243c879d9ff39dd2dcfa Gerrit-Change-Number: 19850 Gerrit-PatchSet: 17 Gerrit-Owner: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Anonymous Coward <lipeng...@apache.org> Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
