[
https://issues.apache.org/jira/browse/HIVE-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675520#comment-15675520
]
Fei Hui commented on HIVE-15221:
--------------------------------
hi [~zsombor.klara]
You said "but a call to System.gc will not guarantee a garbage collection
cycle".
It is related to jvm args DisableExplicitGC. In openjdk7 and openjdk8 default
value is '-XX:-DisableExplicitGC', so garbage collection must occur before
application code running.
in most cases users will not set the arg '-XX:-DisableExplicitGC'.maybe users
know little about it.
So i think it's very userful for users running a job successfully
> Improvement for MapJoin checkMemoryStatus, adding gc before throwing Exception
> ------------------------------------------------------------------------------
>
> Key: HIVE-15221
> URL: https://issues.apache.org/jira/browse/HIVE-15221
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 2.1.0, 2.0.1
> Reporter: Fei Hui
> Assignee: Fei Hui
> Attachments: HIVE-15221.1.patch
>
>
> i see in the current master version
> percentage = (double) usedMemory / (double) maxHeapSize;
> if percentage > maxMemoryUsage, then throw MapJoinMemoryExhaustionException
> in my opinion, running is better than fail. after System.gc, ' if percentage
> > maxMemoryUsage, then throw MapJoinMemoryExhaustionException' maybe better
> And original checking way has a problem: 1) consuming much memory cause gc
> (e.g young gc), then check after adding row and pass. 2) consuming much
> memory does not cause gc, then check after adding rows but throw Exception
> sometimes 2) occurs, but it contians less rows than 1).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
