[
https://issues.apache.org/jira/browse/SPARK-11457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985187#comment-14985187
]
Apache Spark commented on SPARK-11457:
--------------------------------------
User 'jerryshao' has created a pull request for this issue:
https://github.com/apache/spark/pull/9412
> Yarn AM proxy filter configuration should be reloaded when recovered from
> checkpoint
> ------------------------------------------------------------------------------------
>
> Key: SPARK-11457
> URL: https://issues.apache.org/jira/browse/SPARK-11457
> Project: Spark
> Issue Type: Bug
> Components: Streaming, YARN
> Affects Versions: 1.5.1
> Reporter: Saisai Shao
>
> Currently Yarn AM proxy filter configuration is recovered from checkpoint
> file when Spark Streaming application is restarted, which will lead to some
> unwanted behaviors:
> 1. Wrong RM address if RM is redeployed from failure.
> 2. Wrong proxyBase, since app id is updated, old app id for proxyBase is
> wrong.
> So instead of recovering from checkpoint file, these configurations should be
> reloaded each time when app started.
> This problem only exists in Yarn cluster mode, for Yarn client mode, these
> configurations will be updated with RPC message {{AddWebUIFilter}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
