This is an automated email from the ASF dual-hosted git repository. yao pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push: new 254d634548a0 [SPARK-46330] Loading of Spark UI blocks for a long time when HybridStore enabled 254d634548a0 is described below commit 254d634548a0966e3cb501cf580f14c6d2e24732 Author: zhouyifan279 <zhouyifan...@gmail.com> AuthorDate: Wed Dec 20 16:50:38 2023 +0800 [SPARK-46330] Loading of Spark UI blocks for a long time when HybridStore enabled ### What changes were proposed in this pull request? Move `LoadedAppUI` invalidate operation out of `FsHistoryProvider` synchronized block. ### Why are the changes needed? When closing a HybridStore of a `LoadedAppUI` with a lot of data waiting to be written to disk, loading of other Spark UIs will be blocked for a long time. See more details at https://issues.apache.org/jira/browse/SPARK-46330 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Passed existing tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #44260 from zhouyifan279/SPARK-46330. Authored-by: zhouyifan279 <zhouyifan...@gmail.com> Signed-off-by: Kent Yao <y...@apache.org> (cherry picked from commit cf54e8f9a51bf54e8fa3e1011ac370e46134b134) Signed-off-by: Kent Yao <y...@apache.org> --- .../org/apache/spark/deploy/history/FsHistoryProvider.scala | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala index 49b479f3124e..387bc7d9e45b 100644 --- a/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala +++ b/core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala @@ -925,11 +925,12 @@ private[history] class FsHistoryProvider(conf: SparkConf, clock: Clock) * UI lifecycle. */ private def invalidateUI(appId: String, attemptId: Option[String]): Unit = { - synchronized { - activeUIs.get((appId, attemptId)).foreach { ui => - ui.invalidate() - ui.ui.store.close() - } + val uiOption = synchronized { + activeUIs.get((appId, attemptId)) + } + uiOption.foreach { ui => + ui.invalidate() + ui.ui.store.close() } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org