[jira] [Commented] (SPARK-2582) Make Block Manager Master pluggable
[ https://issues.apache.org/jira/browse/SPARK-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068238#comment-14068238 ] Apache Spark commented on SPARK-2582: - User 'harishreedharan' has created a pull request for this issue: https://github.com/apache/spark/pull/1506 > Make Block Manager Master pluggable > --- > > Key: SPARK-2582 > URL: https://issues.apache.org/jira/browse/SPARK-2582 > Project: Spark > Issue Type: Improvement > Components: Block Manager >Affects Versions: 1.0.0 >Reporter: Hari Shreedharan > > Today, there is no way to make the BMM pluggable. So if we want an HA BMM, > that needs to replace the current one. Making this pluggable and selected > based on a config makes it easy to select HA or non-HA one based on the > application's preference. Streaming applications would be better off with an > HA one, while a normal application would not care (since the RDDs can be > regenerated). > Since communication from the Block Managers to the BMM is via akka, we can > keep that the same and just have the implementation of the BMM implement the > actual methods which do the real work - this would not affect the Block > Managers too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-2582) Make Block Manager Master pluggable
[ https://issues.apache.org/jira/browse/SPARK-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068237#comment-14068237 ] Hari Shreedharan commented on SPARK-2582: - PR: https://github.com/apache/spark/pull/1506 > Make Block Manager Master pluggable > --- > > Key: SPARK-2582 > URL: https://issues.apache.org/jira/browse/SPARK-2582 > Project: Spark > Issue Type: Improvement > Components: Block Manager >Affects Versions: 1.0.0 >Reporter: Hari Shreedharan > > Today, there is no way to make the BMM pluggable. So if we want an HA BMM, > that needs to replace the current one. Making this pluggable and selected > based on a config makes it easy to select HA or non-HA one based on the > application's preference. Streaming applications would be better off with an > HA one, while a normal application would not care (since the RDDs can be > regenerated). > Since communication from the Block Managers to the BMM is via akka, we can > keep that the same and just have the implementation of the BMM implement the > actual methods which do the real work - this would not affect the Block > Managers too. -- This message was sent by Atlassian JIRA (v6.2#6252)
