Adrian Bridgett created SPARK-18611:
---------------------------------------
Summary: mesos shuffle service on v2 isn't compatible with spark v1
Key: SPARK-18611
URL: https://issues.apache.org/jira/browse/SPARK-18611
Project: Spark
Issue Type: Bug
Components: Shuffle
Reporter: Adrian Bridgett
Priority: Minor
In SPARK-12583 heartbeating was added to the shuffle service.
This unfortunately changes registerDriver with a different signature so we
can't use sparkv1 clients with sparkv2 mesos shuffle service.
http://apache-spark-developers-list.1001551.n3.nabble.com/YARN-Shuffle-service-and-its-compatibility-td17222.html
is a slightly different issue, but it does suggest that it's agreed that
compatibility is a good idea in these cases.
https://github.com/apache/spark/pull/13279 was created a while ago to backport
this to 1.6.x but was rejected (too invasive). However I don't believe this
issue (v1/v2 compatibility) was recognised back then so perhaps it's time to
reconsider?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
