It's used to tell mesos where is located the spark assembly -- which contains a Mesos Executor implementation (the entry point being spark-executor under the sbin folder).
In a nutshell, the Spark Mesos Framework (see the MesosSchedulerBackend and MesosExecutorBackend impl) will send this URI to the Mesos Master (along with the command ./sbin/spark-executor) for us. These information will be used by the Mesos Slaves to 1/ download the tar (and expand it) 2/ run the command using this expanded folder as CWD. And this command to "start spark" which will run the tasks in the tasks set (from the driver). So this env var is used by these guys, however we prefer to set "spark.executor.uri" in our SparkConf so that it's very specific to the job and easier to control. hth Andy Petrella Belgium (Liège) * ********* Data Engineer in *NextLab <http://nextlab.be/> sprl* (owner) Engaged Citizen Coder for *WAJUG <http://wajug.be/>* (co-founder) Author of *Learning Play! Framework 2 <http://www.packtpub.com/learning-play-framework-2/book>* Bio: on visify <https://www.vizify.com/es/52c3feec2163aa0010001eaa> * ********* Mobile: *+32 495 99 11 04* Mails: - [email protected] - [email protected] * ********* Socials: - Twitter: https://twitter.com/#!/noootsab - LinkedIn: http://be.linkedin.com/in/andypetrella - Blogger: http://ska-la.blogspot.com/ - GitHub: https://github.com/andypetrella - Masterbranch: https://masterbranch.com/andy.petrella On Thu, Apr 3, 2014 at 3:05 PM, felix <[email protected]> wrote: > I deploy spark on mesos successfully, by copying the spark tarball to all > the > slave nodes. I have tried to remove the SPARK_EXECUTOR_URI settings in > spark-env.sh and every things goes well. so , I want to know what does the > SPARK_EXECUTOR_URI setting do, spark 0.9.1 don't use it any more or it's my > misunderstanding. thank you for your reply in advance. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/what-does-SPARK-EXECUTOR-URI-in-spark-env-sh-do-tp3708.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
