Zoltán Borók-Nagy created IMPALA-11428:
------------------------------------------
Summary: Honour Iceberg sort orders when writing a table
Key: IMPALA-11428
URL: https://issues.apache.org/jira/browse/IMPALA-11428
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Zoltán Borók-Nagy
Iceberg specification defines {{{}sort orders{}}}. We should consider this when
writing to an Iceberg table through Impala.
See: [https://iceberg.apache.org/spec/#sort-orders]
Currently we only add Iceberg partition expressions during INSERTs:
[https://github.com/apache/impala/blob/26438d8e3e2cecfdab82643fcee7553df50198ca/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java#L866]
There might be some discrepancy between Impala's and Iceberg's sort orders.
E.g. Impala always sort the files by ascending order, while Iceberg let users
specify the direction. It also let users specify NULLS first vs NULLS last.
Z-order is not part of the Iceberg-spec currently, but it will be in the
future. There will be differences there as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
