Ah, forgot the -verbose option. Thanks Andrew. That is very helpful. Date: Wed, 21 May 2014 11:07:55 -0700 Subject: Re: Is spark 1.0.0 "spark-shell --master=yarn" running in yarn-cluster mode or yarn-client mode? From: and...@databricks.com To: user@spark.apache.org
The answer is actually yarn-client. A quick way to find out: $ bin/spark-shell --master yarn --verbose >From the system properties you can see spark.master is set to "yarn-client." >From the code, this is because args.deployMode is null, and so it's not equal >to "cluster" and so it falls into the second "if" case you mentioned: if (args.deployMode != "cluster" && args.master.startsWith("yarn")) { args.master = "yarn-client"} 2014-05-21 10:57 GMT-07:00 Andrew Lee <alee...@hotmail.com>: Does anyone know if: ./bin/spark-shell --master yarn is running yarn-cluster or yarn-client by default? Base on source code: ./core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala if (args.deployMode == "cluster" && args.master.startsWith("yarn")) { args.master = "yarn-cluster" } if (args.deployMode != "cluster" && args.master.startsWith("yarn")) { args.master = "yarn-client" } It looks like the answer is yarn-cluster mode. I want to confirm this with the community, thanks.