Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22500
Change subject: IMPALA-13768: Redundant Iceberg delete records are shuffled around which cause error "Invalid file path arrived at builder" ...................................................................... IMPALA-13768: Redundant Iceberg delete records are shuffled around which cause error "Invalid file path arrived at builder" IcebergDeleteBuilder assumes that it should only receive delete records for paths of data files that are scheduled for its corresponding SCAN operator. It is not true in the following cases: * number of output channels in sender is 1 (currently no DIRECTED mode, no filtering) * bug in DIRECTED mode, see below * single node plan (no DIRECTED mode, no filtering) With this patch, KrpcDataStreamSender::Send() will use DIRECTED mode even if number of output channels is 1. It also fixes the bug in DIRECTED mode (which was due to an unused variable 'skipped_prev_row') and simplified the logic a bit. The patch also relaxes the assumption in IcebergDeleteBuilder, i.e. only return error for dangling delete records when we are in a distributed plan where we can assume DIRECTED distribution mode of position delete records. Testing * added e2e tests Change-Id: I695c919c9a74edec768e413a02b2ef7dbfa0d6a5 --- M be/src/exec/iceberg-delete-builder.cc M be/src/runtime/krpc-data-stream-sender.cc M testdata/workloads/functional-query/queries/QueryTest/iceberg-delete-partitioned.test 3 files changed, 51 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/22500/1 -- To view, visit http://gerrit.cloudera.org:8080/22500 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I695c919c9a74edec768e413a02b2ef7dbfa0d6a5 Gerrit-Change-Number: 22500 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
