[
https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-6270:
---------------------------------
Description:
If the event logging is enabled, the Spark Standalone Master tries to recreate
the web UI of a completed Spark application from its event logs. However if
this event log is huge (e.g. for a Spark Streaming application), then the
master hangs in its attempt to read and recreate the web ui. This hang causes
the whole standalone cluster to be unusable.
Workaround is to disable the event logging.
was:
If the event logging is enabled, the Spark Standalone Master tries to recreate
the web UI of a completed Spark application from its event logs. However if
this event log is huge (e.g. for a Spark Streaming application), then the
master hangs in its attempt to read and recreate the web ui. This hang causes
the whole standalone cluster to be unusable.
Workaround is to disable the event logging.
[~joshrosen] Another user other than Netflix reported this.
> Standalone Master hangs when streaming job completes
> ----------------------------------------------------
>
> Key: SPARK-6270
> URL: https://issues.apache.org/jira/browse/SPARK-6270
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Streaming
> Affects Versions: 1.2.0, 1.3.0, 1.2.1
> Reporter: Tathagata Das
> Priority: Critical
>
> If the event logging is enabled, the Spark Standalone Master tries to
> recreate the web UI of a completed Spark application from its event logs.
> However if this event log is huge (e.g. for a Spark Streaming application),
> then the master hangs in its attempt to read and recreate the web ui. This
> hang causes the whole standalone cluster to be unusable.
> Workaround is to disable the event logging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
