[
https://issues.apache.org/jira/browse/IMPALA-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058255#comment-18058255
]
ASF subversion and git services commented on IMPALA-14555:
----------------------------------------------------------
Commit 1a814f5f01f1f401517f252aeefe906a2ef1c87a in impala's branch
refs/heads/master from Arnab Karmakar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1a814f5f0 ]
IMPALA-14555: Add Iceberg support for SHOW PARTITIONS WHERE
This patch adds WHERE clause support for SHOW PARTITIONS on Iceberg tables,
extending the functionality previously added for HDFS tables in IMPALA-14065.
Iceberg uses manifest-based file/partition filtering instead of filesystem
traversal. This initial implementation supports simple predicates
(comparisons, IN, BETWEEN, IS NULL, AND/OR) but not functions, subqueries, or
analytics.
For complex queries with functions, use Iceberg metadata tables.
The following is not supported in SHOW PARTITIONS:
SHOW PARTITIONS functional_parquet.iceberg_partitioned
WHERE upper(action) = 'CLICK';
Use the metadata table instead:
SELECT `partition`
FROM functional_parquet.iceberg_partitioned.`partitions`
WHERE upper(`partition`.action) = 'CLICK';
This change also adds DROP PARTITION coverage for BETWEEN predicates on
Iceberg partition columns in iceberg-drop-partition.test.
Testing:
- Unit tests in AnalyzeDDLTest for valid predicates and error cases
- Positive test cases for simple predicates, BETWEEN, IN, IS NULL, AND, OR,
NOT, and boolean literals
- Negative test cases that reject unsupported functions with clear errors
- DROP PARTITION tests with BETWEEN predicates in
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
TODO:
IMPALA-14675: Support complex predicates (SQL functions, non-deterministic
functions, aggregates, analytics, subqueries) in SHOW PARTITIONS WHERE for
Iceberg tables by evaluating them per-partition using
FeSupport.EvalPredicateBatch(), similar to HDFS tables.
Generated-by: Cursor AI (claude-4.5-sonnet)
Change-Id: I4c0ee4d171ae939770725d89dc504e13f82a7688
Reviewed-on: http://gerrit.cloudera.org:8080/23800
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Support Iceberg tables in SHOW PARTITIONS WHERE
> -----------------------------------------------
>
> Key: IMPALA-14555
> URL: https://issues.apache.org/jira/browse/IMPALA-14555
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Arnab Karmakar
> Assignee: Arnab Karmakar
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 5.0.0
>
>
> * Support for applying a {{WHERE}} clause in {{SHOW PARTITIONS}} was
> introduced in IMPALA-14065. However, the implementation was scoped only to
> HDFS-backed tables, and Iceberg support was deferred for a follow-up Jira.
> * The core reason is architectural: Iceberg stores partition metadata in
> {*}manifest files{*}, not in filesystem directory structures as HDFS tables
> do.
> * Currently, running {{SHOW PARTITIONS WHERE}} against an Iceberg table
> results in the following analysis exception:
> {code:java}
> WHERE clause in SHOW PARTITIONS is only supported for HDFS tables {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
