Make sure you are running the MesosShuffleService and not the standard shuffle service:
* org.apache.spark.deploy.mesos.MesosExternalShuffleService v.s. org.apache.spark.deploy.ExternalShuffleService * start-mesos-shuffle-service.sh v.s. start-shuffle-service.sh Thanks to Timothy Chen for the hint. - Eugen Feller