Chao Sun created HIVE-16337: ------------------------------- Summary: HoS: use separate config for mapjoin hash table size limit rather than hive.auto.convert.join.noconditionaltask.size Key: HIVE-16337 URL: https://issues.apache.org/jira/browse/HIVE-16337 Project: Hive Issue Type: Improvement Components: Spark Reporter: Chao Sun
HoS currently share the same config {{hive.auto.convert.join.noconditionaltask.size}} with MR, however it interprets it differently: in MR this is the file size of the input, while in Spark it is the data size. This could cause Spark to be over conservative in many mapjoin optimizations. I suggest we use a separate config for HoS for this purpose. -- This message was sent by Atlassian JIRA (v6.3.15#6346)