[
https://issues.apache.org/jira/browse/IMPALA-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18077415#comment-18077415
]
ASF subversion and git services commented on IMPALA-14913:
----------------------------------------------------------
Commit 8d4578e932c7b76b9f41728b69534f38ed3d33d5 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8d4578e93 ]
IMPALA-14913: Fix partition-related issues of DELETE and UPDATE on Iceberg V3
tables
The following cases were problematic in the original patches:
* partition evolution
* updating partition fields
* predicate pushdown
This patch fixes the above issues. It adds a new method to
DmlStatementBase: getShuffleExprs(): this way we can make shuffling
logic and partition writing logic independent of each other. It is
required for Iceberg V3 tables to write Deletion Vectors correctly:
* even non-partitioned table writes require shuffling
* deletes and updates on partitioned tables require shuffling based
on the deleted records' partition fields, this way we can guarantee
that only one DV is created per data file
Prior to this patch the Deletion Vectors written by Impala did not have
partition information, because of this predicate pushdown filtered them
out even if their corresponding data file passed the filter.
Testing
Added tests from the Iceberg V2 DELETE/UPDATE test cases, they include:
* UPDATE FROM statements
* Updating all types
* Partition transforms
* Partition evolution
* Predicate pushdown
* UDF in the target value
* Constrained memory
Change-Id: I8935d2aac0a8e7776ec7e185e361671054a83227
Reviewed-on: http://gerrit.cloudera.org:8080/24250
Reviewed-by: Peter Rozsa <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> UPDATE statement for Iceberg V3 tables
> --------------------------------------
>
> Key: IMPALA-14913
> URL: https://issues.apache.org/jira/browse/IMPALA-14913
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Implement UPDATE statement for Iceberg V3 tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
