Srikanth Sundarrajan created HIVE-5165:
------------------------------------------
Summary: Evaluate the possibility of enabling feature similar to
pinning tables in memory
Key: HIVE-5165
URL: https://issues.apache.org/jira/browse/HIVE-5165
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Srikanth Sundarrajan
There are many queries involving joins to small tables (ex. dimension tables).
These can be loaded in memory in the map phase to avoid using reduce for the
join operation, but this requires that adequate memory be available to each map
to load the table contents. Further if more small related tables need to be
loaded to realize a join, memory foot print increases, forcing the system to
perhaps consider reduce side join. This issue is to consider the possibility of
using something similar to
{code}
CREATE TABLE t (i INT) ENGINE = MEMORY;
{code}
in mysql, where all relatively small tables (< million rows or < 1G) can be
loaded in memory in a service that the map phase can leverage (over the
network).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
