[ https://issues.apache.org/jira/browse/SPARK-11457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-11457: ------------------------------------ Assignee: (was: Apache Spark) > Yarn AM proxy filter configuration should be reloaded when recovered from > checkpoint > ------------------------------------------------------------------------------------ > > Key: SPARK-11457 > URL: https://issues.apache.org/jira/browse/SPARK-11457 > Project: Spark > Issue Type: Bug > Components: Streaming, YARN > Affects Versions: 1.5.1 > Reporter: Saisai Shao > > Currently Yarn AM proxy filter configuration is recovered from checkpoint > file when Spark Streaming application is restarted, which will lead to some > unwanted behaviors: > 1. Wrong RM address if RM is redeployed from failure. > 2. Wrong proxyBase, since app id is updated, old app id for proxyBase is > wrong. > So instead of recovering from checkpoint file, these configurations should be > reloaded each time when app started. > This problem only exists in Yarn cluster mode, for Yarn client mode, these > configurations will be updated with RPC message {{AddWebUIFilter}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org