[
https://issues.apache.org/jira/browse/IMPALA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18074862#comment-18074862
]
ASF subversion and git services commented on IMPALA-14825:
----------------------------------------------------------
Commit 6128f6b18ecaad8ec7c6f99048d4d28cbf01e6bd in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6128f6b18 ]
IMPALA-14825: Add syntactic sugar for Iceberg Row Lineage data
Currently Row Lineage fields can only be queried by complex expressions:
* row-id:
COALESCE(_file_row_id,
ICEBERG__FIRST__ROW__ID + FILE__POSITION)
* last-updated-sequence-number
COALESCE(_file_last_updated_sequence_number,
ICEBERG__DATA__SEQUENCE__NUMBER)
This patch adds _row_id and _last_updated_sequence_number virtual
columns to Iceberg V3 tables as syntactic sugars, i.e. they get
re-written during analysis to the corresponding COALESCE() expr.
_row_id and _last_updated_sequence_number are being used by other
engines as well, e.g. Spark.
Testing:
* e2e tests added
* column-masking tests
Generated-by: Claude Sonnet 4.6
Generated-by: Claude Opus 4.6
Change-Id: I02aceeb0969243ba6d28e4c0c97d5dd7a5afd765
Reviewed-on: http://gerrit.cloudera.org:8080/24184
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add syntactic sugar for Iceberg Row Lineage data
> ------------------------------------------------
>
> Key: IMPALA-14825
> URL: https://issues.apache.org/jira/browse/IMPALA-14825
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Currently Row Lineage fields can be queried by complex expressions:
> * row-id
> **
> {noformat}
> COALESCE(_file_row_id, ICEBERG__FIRST__ROW__ID + FILE__POSITION)
> {noformat}
> * last-updated-sequence-number
> **
> {noformat}
> COALESCE(_file_last_updated_sequence_number, ICEBERG__DATA__SEQUENCE__NUMBER)
> {noformat}
> We could add ICEBERG\_\_ROW\_\_ID and
> ICEBERG\_\_LAST\_\_UPDATED\_\_SEQUENCE\_\_NUMBER virtual columns as
> placeholders that would always get re-written during analysis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
