Selina Zhang created HIVE-9573: ---------------------------------- Summary: Lazy load partitions for SELECT LIMIT type query Key: HIVE-9573 URL: https://issues.apache.org/jira/browse/HIVE-9573 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Selina Zhang Assignee: Selina Zhang
Some tools such as HUE uses SELECT * FROM table LIMIT 100; to grab the sample content of table. For table with large partitions it causes a large amount of partition objects been loaded. Thus slow down the HS2 even cause OOM from time to time. My solution is lazy load partition objects in FetchOperator for this type of query. We can skip retrieving the whole partitions but only partition names in PartitionPrunner, and lazy load the partition objects when needed in execution time (for local job only). I have a patch ready. But want to hear more suggestions. Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)