[
https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749610#comment-15749610
]
Marcelo Vanzin commented on SPARK-18085:
----------------------------------------
bq. while I pressed a link on SHS application page
That by itself proves you're not talking about data that is coming from the
event log. The SHS *does not store the event log in memory*. It has its own
internal representation of data.
bq. (or UI response in the SHS memory)
YES, "UI response". That is *not* the event log. The event log being in JSON
has absolute zero to do with the fact that responses built by the Spark UI take
a lot of memory.
I'm gonna say again: your points are valid but unrelated to this work. If you
keep insisting on talking about unrelated issues as part of this work, I'll
just keep saying that you should file other bugs to track those issues, because
they're completely unrelated to what is being proposed here, and discussing
them here is just a distraction and is not going to lead to the issues you care
about being fixed.
> Better History Server scalability for many / large applications
> ---------------------------------------------------------------
>
> Key: SPARK-18085
> URL: https://issues.apache.org/jira/browse/SPARK-18085
> Project: Spark
> Issue Type: Umbrella
> Components: Spark Core, Web UI
> Affects Versions: 2.0.0
> Reporter: Marcelo Vanzin
> Attachments: spark_hs_next_gen.pdf
>
>
> It's a known fact that the History Server currently has some annoying issues
> when serving lots of applications, and when serving large applications.
> I'm filing this umbrella to track work related to addressing those issues.
> I'll be attaching a document shortly describing the issues and suggesting a
> path to how to solve them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
