Re: problem about cluster mode of spark 1.0.0

2014-06-24 Thread Andrew Or
Hi Randy and Gino, The issue is that standalone-cluster mode is not officially supported. Please use standalone-client mode instead, i.e. specify --deploy-mode client in spark-submit, or simply leave out this config because it defaults to client mode. Unfortunately, this is not currently

Re: problem about cluster mode of spark 1.0.0

2014-06-24 Thread Gino Bustelo
Andrew, Thanks for your answer. It validates our finding. Unfortunately, client mode assumes that I'm running in a privilege node. What I mean by privilege is a node that has net access to all the workers and vice versa. This is a big assumption to make and unreasonable in certain

problem about cluster mode of spark 1.0.0

2014-06-20 Thread randylu
my programer runs in standalone model, the commond line is likeļ¼š /opt/spark-1.0.0/bin/spark-submit \ --verbose \ --class $class_name --master spark://master:7077 \ --driver-memory 15G \ --driver-cores 2 \ --deploy-mode cluster \