I'm loading these settings from a properties file: spark.executor.memory=256M spark.cores.max=1 spark.shuffle.consolidateFiles=true spark.task.cpus=1 spark.deploy.defaultCores=1 spark.driver.cores=1 spark.scheduler.mode=FAIR
Once the job is submitted to mesos, I can go to the spark UI for that job (hostname:4040) and on the environment tab. I see that those settings are there. If I then comment out all those settings and allow spark to use the defaults, it still appears to use the same settings in mesos. Under both runs, it still shows 1 task, 3 cpu, 1GB memory. Nothing seems to change no matter what is put in that props file, even if they show up in the spark environment tab.
