Re: Getting the number of slaves

2015-07-30 Thread Tathagata Das
ache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p24043.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.

Re: Getting the number of slaves

2015-07-28 Thread amkcom
try sc.getConf.getInt("spark.executor.instances", 1) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p24043.html Sent from the Apache Spark User List mailing list archive at

Re: Getting the number of slaves

2014-07-30 Thread Sung Hwan Chung
I should have searched more... > >>> > >>> Something I noticed though: after the SparkContext is initialized, I > had > >>> to > >>> wait for a few seconds until sc.getExecutorStorageStatus.length returns > >>> the > >>> correct number of workers

Re: Getting the number of slaves

2014-07-28 Thread Zongheng Yang
rs in my cluster (otherwise it returns 1, for the >>> driver)... >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p10619.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> >

Re: Getting the number of slaves

2014-07-28 Thread Andrew Or
turns 1, for the >> driver)... >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p10619.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >

Re: Getting the number of slaves

2014-07-28 Thread Sung Hwan Chung
or the > driver)... > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p10619.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Getting the number of slaves

2014-07-24 Thread Nicolas Mai
)... -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604p10619.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Getting the number of slaves

2014-07-24 Thread Evan R. Sparks
ntext: > http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Getting the number of slaves

2014-07-24 Thread Nicolas Mai
currently in use, in order set the level of parallelism and RDD partitions, based on that number. Thanks! Nicolas -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-number-of-slaves-tp10604.html Sent from the Apache Spark User List mailing li