Saisai Shao created SPARK-11457:
-----------------------------------
Summary: Yarn AM proxy filter configuration should be reloaded
when recovered from checkpoint
Key: SPARK-11457
URL: https://issues.apache.org/jira/browse/SPARK-11457
Project: Spark
Issue Type: Bug
Components: Streaming, YARN
Affects Versions: 1.5.1
Reporter: Saisai Shao
Currently Yarn AM proxy filter configuration is recovered from checkpoint file
when Spark Streaming application is restarted, which will lead to some unwanted
behaviors:
1. Wrong RM address if RM is redeployed from failure.
2. Wrong proxyBase, since app id is updated, old app id for proxyBase is wrong.
So instead of recovering from checkpoint file, these configurations should be
reloaded each time when app started.
This problem only exists in Yarn cluster mode, for Yarn client mode, these
configurations will be updated with RPC message {{AddWebUIFilter}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
