[
https://issues.apache.org/jira/browse/HIVE-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279770#comment-14279770
]
Rui Li commented on HIVE-9372:
------------------------------
Checked the log of failed tests. mapjoin_memcheck failed due to:
{noformat}
2015-01-15 03:06:58,233 ERROR [main]: mr.MapredLocalTask
(MapredLocalTask.java:executeInProcess(350)) - Hive Runtime Error: Map local
work exhausted memory
org.apache.hadoop.hive.ql.exec.mapjoin.MapJoinMemoryExhaustionException:
2015-01-15 03:06:58 Processing rows: 2 Hashtable size: 1
Memory usage: 1724271072 percentage: 0.903
{noformat}
The patch is unlikely to introduce a memory leak and the test data size is
quite small so we're actually launching one thread.
I cannot reproduce udaf_percentile_approx_23 on my side.
> Parallel checking non-combinable paths in CombineHiveInputFormat
> ----------------------------------------------------------------
>
> Key: HIVE-9372
> URL: https://issues.apache.org/jira/browse/HIVE-9372
> Project: Hive
> Issue Type: Improvement
> Reporter: Rui Li
> Assignee: Rui Li
> Attachments: HIVE-9372.1.patch, HIVE-9372.2.patch,
> HIVE-9372.3-spark.patch
>
>
> Checking if an input path is combinable is expensive. So we should make it
> parallel.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
