[
https://issues.apache.org/jira/browse/IMPALA-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822645#comment-17822645
]
Daniel Becker commented on IMPALA-12845:
----------------------------------------
The problem is that for Iceberg tables, the DESCRIBE statement returns four
columns: {{name, type, comment and nullable}} (only Iceberg and Kudu tables
have '{{{}nullable{}}}'). The describe statement for complex types only returns
the first three columns as they are always nullable.
For non-Iceberg non-Kudu tables and complex types see
[https://github.com/apache/impala/blob/30fbcc94ea08bb0a5eb58d68bcce9c2a0eb9750e/fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java#L269|https://github.com/apache/impala/blob/30fbcc94ea08bb0a5eb58d68bcce9c2a0eb9750e/fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java#L269,],
for Iceberg tables see
[https://github.com/apache/impala/blob/30fbcc94ea08bb0a5eb58d68bcce9c2a0eb9750e/fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java#L334.]
The DCHECK in HS2ColumnarResultSet::AddOneRow() expects the number of columns
to be the same from the DESCRIBE statement response and the {{metadata_}}
field, for which the columns are ultimately set here:
[https://github.com/apache/impala/blob/2f14fd29c0b47fc2c170a7f0eb1cecaf6b9704f4/fe/src/main/java/org/apache/impala/service/Frontend.java#L661]
I'll try to find a way to skip adding "nullable" if the target is a complex
type instead of a table.
> Crash with DESCRIBE on a struct from an Iceberg table
> -----------------------------------------------------
>
> Key: IMPALA-12845
> URL: https://issues.apache.org/jira/browse/IMPALA-12845
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Major
>
> A DESCRIBE statement on a struct contained in an Iceberg table crashes Impala:
> In Hive:
> {code:java}
> create table ice_struct (id INT, s STRUCT<i: INT>) stored by iceberg;{code}
> In Impala:
> {code:java}
> describe ice_struct.s
> Caught exception TSocket read 0 bytes, type=<class
> 'thrift.transport.TTransport.TTransportException'> in FetchResults.
> Error communicating with impalad: TSocket read 0 bytes{code}
> The logs in impalad.FATAL indicate that we hit a DCHECK:
> {code:java}
> F0227 11:53:49.936864 680819 query-result-set.cc:386] Check failed: num_col
> == metadata_.columns.size() (3 vs. 4){code}
>
> Note that this also happens with Iceberg metadata tables, for example:
> {code:java}
> describe
> functional_parquet.iceberg_query_metadata.entries.readable_metrics;{code}
> With non-Iceberg tables there is no error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
