Miklos Szurap created SPARK-39222:
-------------------------------------
Summary: Add JvmPauseMonitor to Spark History Server
Key: SPARK-39222
URL: https://issues.apache.org/jira/browse/SPARK-39222
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.2.1, 3.0.0, 2.4.8, 2.0.0
Reporter: Miklos Szurap
It is hard to understand what happens with Spark History Server when long JVM
pauses (GC or host level pauses) are causing unresponsiveness.
Similar to Hadoop's implementation (initially in HADOOP-9618) it would be
beneficial to add the
[JVMPauseMonitor|https://github.com/apache/hadoop/blob/f4b24c68e76df40d55258fc5391baabfa9ac362d/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java]
to the HistoryServer (HistoryServerSuite.scala).
This will make GC pauses obvious in logs and lets administrators easily notice
it and react in time - adjust configurations with increasing the SHS heap size.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
