Re: Shuffle service fails to register driver - Spark - Mesos

Make sure you are running the MesosShuffleService and not the standard shuffle service: * org.apache.spark.deploy.mesos.MesosExternalShuffleService v.s. org.apache.spark.deploy.ExternalShuffleService * start-mesos-shuffle-service.sh v.s. start-shuffle-service.sh Thanks to Timothy Chen

Re: Shuffle service fails to register driver - Spark - Mesos

Never mind, just figured out my problem, I was running: *deploy.ExternalShuffleService* instead of *deploy.mesos.MesosExternalShuffleService* - Jo Voordeckers On Fri, Apr 15, 2016 at 2:29 PM, Jo Voordeckers wrote: > Forgot to mention we're running Spark (Streaming)

Re: Shuffle service fails to register driver - Spark - Mesos

Forgot to mention we're running Spark (Streaming) 1.5.1 - Jo Voordeckers On Fri, Apr 15, 2016 at 12:21 PM, Jo Voordeckers wrote: > Hi all, > > I've got mesos in coarse grained mode with dyn alloc, shuffle service > enabled and am running the shuffle service on every

Shuffle service fails to register driver - Spark - Mesos

Hi all, I've got mesos in coarse grained mode with dyn alloc, shuffle service enabled and am running the shuffle service on every mesos slave. I'm assuming I misconfigured something on the scheduler service, any ideas? On my driver is see a few of these, I guess it's one for every executor :