On Mon, Apr 18, 2016 at 1:53 PM, Reynold Xin <[email protected]> wrote: > That's not the only one. For example, the hash shuffle manager has been off > by default since Spark 1.2, and we'd like to remove it in 2.0: > https://github.com/apache/spark/pull/12423
If I understand things correctly, Mark's option B (running a single shuffle service, the one from the older Spark release) would still work, wouldn't it? You'd run into problems when Spark adds a new shuffle manager that is not known to the old shuffle service, though. Perhaps at that time we should investigate making the shuffle service more agnostic to the app's shuffle manager. -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
