This is an automated email from the ASF dual-hosted git repository. zabetak pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push: new c126422a91b HIVE-27695: HIVE-26828: Intermittent OOM when running TestMiniTezCliDriver (Stamatis Zampetakis reviewed by Ayush Saxena) c126422a91b is described below commit c126422a91be695c75ec4a750638a0aa4d1ba6cd Author: Stamatis Zampetakis <zabe...@gmail.com> AuthorDate: Wed Oct 11 12:17:23 2023 +0200 HIVE-27695: HIVE-26828: Intermittent OOM when running TestMiniTezCliDriver (Stamatis Zampetakis reviewed by Ayush Saxena) java.lang.OutOfMemoryError: GC overhead limit exceeded is thrown by the Tez Application Master (AM) cause the current heap size (128MB) is not enough to accommodate the needs of multiple Tez containers running. Each running container requires roughly 10MB of memory in the AM. The AM accumulates/manipulates multiple configuration objects (some of them retaining as much as 1MB of heap) per container. The heap gradually becomes full and GC is spending a lot of CPU time to clean things up without really making much progress since containers are reused and heap cannot shrink. There are multiple solutions to the problem but the easiest and most effective is to increase the heap size for the AM. At this point in time, 512MB is a good value. The hybridgrace_hashjoin_2.q test which was failing to due OOM can now be re-enabled. Close apache/hive#4792 --- data/conf/tez/tez-site.xml | 2 +- ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/data/conf/tez/tez-site.xml b/data/conf/tez/tez-site.xml index ba4df319a55..88adb6a57e8 100644 --- a/data/conf/tez/tez-site.xml +++ b/data/conf/tez/tez-site.xml @@ -1,7 +1,7 @@ <configuration> <property> <name>tez.am.resource.memory.mb</name> - <value>128</value> + <value>512</value> </property> <property> <name>tez.task.resource.memory.mb</name> diff --git a/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q b/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q index 93b8c13a49f..6ed771ba685 100644 --- a/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q +++ b/ql/src/test/queries/clientpositive/hybridgrace_hashjoin_2.q @@ -1,7 +1,6 @@ --! qt:dataset:srcpart --! qt:dataset:src1 --! qt:dataset:src ---! qt:disabled:HIVE-26820 Disable hybridgrace_hashjoin_2.q flaky test set hive.mapred.mode=nonstrict; set hive.explain.user=false; -- Hybrid Grace Hash Join