Thanks Josh and Marcelo! It now works! BTW, just wondering, is there any perf difference between running spark in standalone mode and under yarn? The only goal that I created this cluster is to run spark jobs. So I can set up spark in standalone mode if it runs slow in yarn. best. From: Josh Rosen <rosenvi...@gmail.com> To: Marcelo Vanzin <van...@cloudera.com> Cc: freedafeng <freedaf...@yahoo.com>; "user@spark.apache.org" <user@spark.apache.org> Sent: Wednesday, January 14, 2015 3:20 PM Subject: Re: how to run python app in yarn? There's an open PR for supporting yarn-cluster mode in PySpark: https://github.com/apache/spark/pull/3976 (currently blocked on reviewer attention / time)
On Wed, Jan 14, 2015 at 3:16 PM, Marcelo Vanzin <van...@cloudera.com> wrote: As the error message says... On Wed, Jan 14, 2015 at 3:14 PM, freedafeng <freedaf...@yahoo.com> wrote: > Error: Cluster deploy mode is currently not supported for python > applications. Use "yarn-client" instead of "yarn-cluster" for pyspark apps. -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org