Ah, I missed that, thanks! On Mon, Sep 14, 2015 at 11:45 AM, Tim Chen <t...@mesosphere.io> wrote:
> Hi Philip, > > I've included documentation in the Spark/Mesos doc ( > http://spark.apache.org/docs/latest/running-on-mesos.html), where you can > start the MesosShuffleService with sbin/start-mesos-shuffle-service.sh > script. > > The shuffle service needs to be started manually for Mesos on each slave > (one way is via Marathon with unique hostname constraint), and then you > need to enable dynamicAllocation and shuffle service flag on the driver and > it should work. > > Let me know if that's not clear. > > Tim > > On Mon, Sep 14, 2015 at 11:36 AM, Philip Weaver <philip.wea...@gmail.com> > wrote: > >> Hello, I am trying to use dynamic allocation which requires the shuffle >> service. I am running Spark on mesos. >> >> Whenever I set spark.shuffle.service.enabled=true, my Spark driver fails >> with an error like this: >> >> Caused by: java.net.ConnectException: Connection refused: devspark1/ >> 172.26.21.70:7337 >> >> It's not clear from the documentation if the shuffle service starts >> automatically just by having it enabled, or if I need to do something else. >> There are instructions for running the shuffle service in YARN, but not >> mesos. >> >> - Philip >> >> >
