Vineet Garg created HIVE-23050:
----------------------------------
Summary: Partition pruning cache miss during compilation
Key: HIVE-23050
URL: https://issues.apache.org/jira/browse/HIVE-23050
Project: Hive
Issue Type: Bug
Components: Query Planning
Affects Versions: 4.0.0
Reporter: Vineet Garg
Assignee: Vineet Garg
{code:sql}
create table pcr_t1 (key int, value string) partitioned by (ds string);
insert overwrite table pcr_t1 partition (ds='2000-04-08') select * from src
where key < 20 order by key;
insert overwrite table pcr_t1 partition (ds='2000-04-09') select * from src
where key < 20 order by key;
insert overwrite table pcr_t1 partition (ds='2000-04-10') select * from src
where key < 20 order by key;
explain extended select key, value, ds from pcr_t1 where (ds='2000-04-08' and
key=1) or (ds='2000-04-09' and key=2) order by key, value, ds
{code}
During query compilation HivePartitionPruner fetches list of partition and
caches it, later PCR (partition condition removal) tries to get pruned
partitions but due to cache miss, request goes to metastore server to retrieve
pruned partitions using listPartitions.
Improvement here would be to use the list of partitions already cached to do
the partition pruning for PCR or pruning in general
(I am not sure why HivePartitionPruner isn't able to do partition pruning in
the first place)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
