Hi there, I am doing a POC with Spark and I have noticed that if I run my job on standalone spark installation, it finishes in a second(It's a small sample job). But when I run same job on spark cluster with Yarn, it takes 4-5 min in simple execution. Are there any best practices that I need to follow for spark cluster configuration. I have left all default settings. During spark-submit I specify num-executors=3, executor-memory=512m, executor-cores-1.
I am using Java Spark SQL API. Thanks, Purvi -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-jobs-run-extremely-slow-on-yarn-cluster-compared-to-standalone-spark-tp26215.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
