[ https://issues.apache.org/jira/browse/HIVE-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15675520#comment-15675520 ]
Fei Hui commented on HIVE-15221: -------------------------------- hi [~zsombor.klara] You said "but a call to System.gc will not guarantee a garbage collection cycle". It is related to jvm args DisableExplicitGC. In openjdk7 and openjdk8 default value is '-XX:-DisableExplicitGC', so garbage collection must occur before application code running. in most cases users will not set the arg '-XX:-DisableExplicitGC'.maybe users know little about it. So i think it's very userful for users running a job successfully > Improvement for MapJoin checkMemoryStatus, adding gc before throwing Exception > ------------------------------------------------------------------------------ > > Key: HIVE-15221 > URL: https://issues.apache.org/jira/browse/HIVE-15221 > Project: Hive > Issue Type: Improvement > Components: Query Processor > Affects Versions: 2.1.0, 2.0.1 > Reporter: Fei Hui > Assignee: Fei Hui > Attachments: HIVE-15221.1.patch > > > i see in the current master version > percentage = (double) usedMemory / (double) maxHeapSize; > if percentage > maxMemoryUsage, then throw MapJoinMemoryExhaustionException > in my opinion, running is better than fail. after System.gc, ' if percentage > > maxMemoryUsage, then throw MapJoinMemoryExhaustionException' maybe better > And original checking way has a problem: 1) consuming much memory cause gc > (e.g young gc), then check after adding row and pass. 2) consuming much > memory does not cause gc, then check after adding rows but throw Exception > sometimes 2) occurs, but it contians less rows than 1). -- This message was sent by Atlassian JIRA (v6.3.4#6332)