specifying worker nodes when using the repl?

Hi I am working with a Cloudera 5 cluster with 192 nodes and can’t work out how to get the spark repo to use more than 2 nodes in an interactive session. So, this works, but is non-interactive (using yarn-client as MASTER)

Re: specifying worker nodes when using the repl?

Hi Eric, Have you tried setting the SPARK_WORKER_INSTANCES env variable before running spark-shell? http://spark.apache.org/docs/0.9.0/running-on-yarn.html -Sandy On Mon, May 19, 2014 at 8:08 AM, Eric Friedman e...@spottedsnake.netwrote: Hi I am working with a Cloudera 5 cluster with 192

Re: specifying worker nodes when using the repl?

Sandy, thank you so much — that was indeed my omission! Eric On May 19, 2014, at 10:14 AM, Sandy Ryza sandy.r...@cloudera.com wrote: Hi Eric, Have you tried setting the SPARK_WORKER_INSTANCES env variable before running spark-shell? http://spark.apache.org/docs/0.9.0/running-on-yarn.html