[
https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053929#comment-15053929
]
Andrew Or commented on SPARK-6270:
----------------------------------
[~shivaram] That may be difficult to do because different applications can
specify different log directories, whereas now the history server reads all
logs in the same one. Also it would add complexity to the Master process
because now it binds to two ports instead of one. I think we should try to keep
it lightweight in the future and simply rip out this functionality. With Spark
2.0 I believe we're allowed to do that. :)
> Standalone Master hangs when streaming job completes and event logging is
> enabled
> ---------------------------------------------------------------------------------
>
> Key: SPARK-6270
> URL: https://issues.apache.org/jira/browse/SPARK-6270
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Streaming
> Affects Versions: 1.2.0, 1.2.1, 1.3.0, 1.5.1
> Reporter: Tathagata Das
> Priority: Critical
>
> If the event logging is enabled, the Spark Standalone Master tries to
> recreate the web UI of a completed Spark application from its event logs.
> However if this event log is huge (e.g. for a Spark Streaming application),
> then the master hangs in its attempt to read and recreate the web ui. This
> hang causes the whole standalone cluster to be unusable.
> Workaround is to disable the event logging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
