okumin commented on code in PR #5562:
URL: https://github.com/apache/hive/pull/5562#discussion_r1862206780
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionHandler.java:
##########
@@ -96,7 +96,12 @@ public void checkMemoryStatus(long tableContainerSize, long
numRows)
+ tableContainerSize + "\tMemory usage:\t" + usedMemory +
"\tpercentage:\t" + percentageNumberFormat.format(percentage);
console.printInfo(msg);
if(percentage > maxMemoryUsage) {
- throw new MapJoinMemoryExhaustionError(msg);
+ System.gc();
Review Comment:
First, I might not have sufficient knowledge to give this +1. I've never
faced this error.
Reading the source code, I found that only Hive on MapReduce uses
`MemoryExhaustionChecker`. So, not only LLAP but also Tez are not involved with
this error. My original concern, i.e., the entire LLAP daemon could get
degraded, is gone.
But I am still wondering why Hive on MapReduce needs
`MemoryExhaustionChecker`. We may remove the class and let MapReduce crush with
OOM when creating a giant hash table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
