[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051438#comment-15051438 ]
Shivaram Venkataraman commented on SPARK-6270: ---------------------------------------------- FWIW this problem happens 1.5.2 as well -- I'm not sure if something in the event logging infrastructure changed but for one our applications we end up getting 2.5G event logs for a short job (around 20mins or so). This in turn leads to https://issues.apache.org/jira/browse/SPARK-11928 > Standalone Master hangs when streaming job completes and event logging is > enabled > --------------------------------------------------------------------------------- > > Key: SPARK-6270 > URL: https://issues.apache.org/jira/browse/SPARK-6270 > Project: Spark > Issue Type: Bug > Components: Deploy, Streaming > Affects Versions: 1.2.0, 1.2.1, 1.3.0, 1.5.1 > Reporter: Tathagata Das > Priority: Critical > > If the event logging is enabled, the Spark Standalone Master tries to > recreate the web UI of a completed Spark application from its event logs. > However if this event log is huge (e.g. for a Spark Streaming application), > then the master hangs in its attempt to read and recreate the web ui. This > hang causes the whole standalone cluster to be unusable. > Workaround is to disable the event logging. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org