[
https://issues.apache.org/jira/browse/HIVE-7469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangmeng updated HIVE-7469:
---------------------------
Description:
In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea
about how to deal with skew join key , the key point is that use mapjoin to
deal with skew key, but there has a case which troubles me:
if the two join tables have the same big skew key on one value :
for example , select * from table A join B on A.id=b.id, both table A and
B have a lot of keys on id=1, in this case , if we use map join to deal
with the skew key id=1 ,maybe it will OOM.
so ,how to fix this case? Will it rollback to common join ? Thanks.
was:
In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea
about how to deal with skew join key , the key is that use mapjoin to deal
with skew key, but there has a case which troubles me:
if the two join tables have the same big skew key on one value :
for example , select * from table A join B on A.id=b.id, both table A and
B have a lot of keys on id=1, in this case , if we use map join to deal
with the skew key id=1 ,maybe it will OOM.
so ,how to fix this case? Will it rollback to common join ? Thanks.
> skew join keys when two join table have the same big skew key
> -----------------------------------------------------------------
>
> Key: HIVE-7469
> URL: https://issues.apache.org/jira/browse/HIVE-7469
> Project: Hive
> Issue Type: Improvement
> Reporter: wangmeng
>
> In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea
> about how to deal with skew join key , the key point is that use mapjoin
> to deal with skew key, but there has a case which troubles me:
> if the two join tables have the same big skew key on one value :
> for example , select * from table A join B on A.id=b.id, both table A
> and B have a lot of keys on id=1, in this case , if we use map join
> to deal with the skew key id=1 ,maybe it will OOM.
> so ,how to fix this case? Will it rollback to common join ? Thanks.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
