[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #35856: [SPARK-38550][SQL][CORE] Use a disk-based store to save more debug information for live UI

2022-04-04 Thread GitBox
dongjoon-hyun commented on code in PR #35856: URL: https://github.com/apache/spark/pull/35856#discussion_r841422549 ## sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala: ## @@ -118,6 +119,12 @@ private[sql] class SharedState( statusStore } +

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #35856: [SPARK-38550][SQL][CORE] Use a disk-based store to save more debug information for live UI

2022-04-04 Thread GitBox
dongjoon-hyun commented on code in PR #35856: URL: https://github.com/apache/spark/pull/35856#discussion_r841421049 ## core/src/main/scala/org/apache/spark/internal/config/Status.scala: ## @@ -70,4 +70,11 @@ private[spark] object Status { .version("3.0.0")

Live UI

2015-10-12 Thread Jakob Odersky
Hi everyone, I am just getting started working on spark and was thinking of a first way to contribute whilst still trying to wrap my head around the codebase. Exploring the web UI, I noticed it is a classic request-response website, requiring manual refresh to get the latest data. I think it

Re: Live UI

2015-10-12 Thread Ryan Williams
Yea, definitely check out Spree <https://github.com/hammerlab/spree>! It functions as "live" UI, history server, and archival storage of event log data. There are pros and cons to building something like it in Spark trunk (and running it in the Spark driver, presumably) that

Re: Live UI

2015-10-12 Thread Holden Karau
I don't think there has been much work done with ScalaJS and Spark (outside of the April fools press release), but there is a live Web UI project out of hammerlab with Ryan Williams https://github.com/hammerlab/spree which you may want to take a look at. On Mon, Oct 12, 2015 at 2:36 PM, Jakob