[
https://issues.apache.org/jira/browse/SPARK-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294311#comment-14294311
]
Imran Rashid commented on SPARK-3454:
-------------------------------------
There are a few ways we could move forward on this.
1. Its pretty easy to have the UI generate some json -- we just need to
implement {{renderJson}} in a bunch of the {{Page}} classes. sarutak's PR came
pretty close to doing this. I have a branch where I merged it w/ master and
filled in a few holes. However, this approach has the downside that we may end
up creating some new semi-public, semi-supported api in the schema for whatever
json we output. Since we're building it on top of the UI layer, it might be
hard to really guarantee we can keep it as a stable api.
2. We could extend {{SparkStatusTracker}} to cover a lot more, and then build
up everything this on top of those events. We'd need to add more events,
especially task metrics & storage info, but probably everything in the UI.
Since we want the json available from the {{HistoryServer}} as well, we might
need to change around the logging to be compatible with {{SparkStatusTracker}}
(and whatever we add to it).
Even the UI could be made a consumer of the json output, as a way to (a) unify
some of the logic in the logging & UI & (b) make the json api a true
first-class api, since we'd be consuming it ourselves.
> Expose JSON representation of data shown in WebUI
> -------------------------------------------------
>
> Key: SPARK-3454
> URL: https://issues.apache.org/jira/browse/SPARK-3454
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 1.1.0
> Reporter: Kousuke Saruta
>
> If WebUI support to JSON format extracting, it's helpful for user who want to
> analyse stage / task / executor information.
> Fortunately, WebUI has renderJson method so we can implement the method in
> each subclass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
