Does anyone know if: ./bin/spark-shell --master yarn is running yarn-cluster or yarn-client by default? Base on source code:
./core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
if (args.deployMode == "cluster" && args.master.startsWith("yarn")) {
args.master = "yarn-cluster" } if (args.deployMode != "cluster" &&
args.master.startsWith("yarn")) { args.master = "yarn-client"
}
It looks like the answer is yarn-cluster mode.
I want to confirm this with the community, thanks.
