Jeffrey(Xilang) Yan created HIVE-20210:
------------------------------------------
Summary: Simple Fetch optimizer should lead to MapReduce when
filter on non-partition column and conversion is minimal
Key: HIVE-20210
URL: https://issues.apache.org/jira/browse/HIVE-20210
Project: Hive
Issue Type: Bug
Components: Query Planning
Affects Versions: 2.3.2, 2.3.1, 2.3.0
Reporter: Jeffrey(Xilang) Yan
Assignee: Jeffrey(Xilang) Yan
When conversion is minimal, simple fetch can be used only when filter on
partition column or no filter at all. But it lead to simple fetch even if
filter on non-partition column. Unit test " select * from srcpart where key >
100 limit 10 " in the nonmr_fetch.q demonstration this issue – the unit test is
not correct indeed(it should be Map Reduce but in test it is Simple Fetch).
This issue lead to a serious problem when data size is huge. When conversion is
more and filter on both partition column and non-partition column, it will not
chech hive.fetch.task.conversion.threshold, which lead to the query to takes
hours to finish. This issue doesn't exist in 1.2.1, how it works should be a
magic...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
