[ https://issues.apache.org/jira/browse/IMPALA-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628693#comment-17628693 ]
Qizhu Chan edited comment on IMPALA-11701 at 11/4/22 6:14 AM: -------------------------------------------------------------- Oh, I found an issue [IMPALA-11171|https://issues.apache.org/jira/browse/IMPALA-11171] that seems to be related to my issue ? was (Author: libra_816): Oh, I found an issue[IMPALA-11171|https://issues.apache.org/jira/browse/IMPALA-11171] that seems to be related to my issue ? > Slow query problem about querying iceberg table by impala > --------------------------------------------------------- > > Key: IMPALA-11701 > URL: https://issues.apache.org/jira/browse/IMPALA-11701 > Project: IMPALA > Issue Type: Question > Reporter: Qizhu Chan > Priority: Major > Labels: impala-iceberg > Attachments: image-2022-11-03-17-37-14-712.png, > profile_cf446a1ab3a5e852_1b1005de00000000.txt > > > I use impala to query iceberg table, but the query efficiency is not ideal, > compared with querying the hive format table of the same data, the > time-consuming increase is dozens of times. > The sql statement used is a very simple statistical query, be like : > select count(*) from `db_name`.tbl_name where datekey='20221001' and > event='xxx' > ('datekey' and 'event' are the partition fields) > My personal feeling is that impala might fetch iceberg's metadata stats and > return results very quickly, but it doesn't. > The catalog of iceberg table is of the hadoop type, and Impala can access it > by creating an external table in hive. By the way, iceberg table will > perform snapshot expiration and data compaction on a daily basis, so there > should be no small file problems. > I found this warning using the explain statement: > {code:java} > | WARNING: The following tables are missing relevant table and/or column > statistics. | > | iceberg.gamebox_event_iceberg > {code} > Query: SHOW TABLE STATS `iceberg`.gamebox_event_iceberg > +-------+--------+--------+--------------+-------------------+---------+-------------------+-----------------------------------------------------------------+ > | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | > Incremental stats | Location > | > +-------+--------+--------+--------------+-------------------+---------+-------------------+-----------------------------------------------------------------+ > | 0 | 590509 | 1.91TB | NOT CACHED | NOT CACHED | PARQUET | > false | hdfs:///hive/warehouse/iceberg/gamebox_event_iceberg | > +-------+--------+--------+--------------+-------------------+---------+-------------------+-----------------------------------------------------------------+ > It seems like Impala is not syncing iceberg's table and column statistics. > I'm not sure if this has anything to do with slow queries. > As shown in the screenshot, i think the query time is mainly on planning and > execution backends , but I don't know what is the reason for these two time > consuming. > Attachment is the complete profile for this query. > How do I speed up the query? Can someone help with my question?plz..... > !image-2022-11-03-17-37-14-712.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org