Just to add you can also look into SPARK_WORKER_INSTANCES configuration in
the spark-env.sh file.
On Aug 17, 2015 3:44 AM, "Daniel Darabos" <daniel.dara...@lynxanalytics.com>
wrote:
> Hi Praveen,
>
> On Mon, Aug 17, 2015 at 12:34 PM, praveen S <mylogi...@gmail.com> wrote:
>
>> What does this mean in .setMaster("local[2]")
>>
> Local mode (executor in the same JVM) with 2 executor threads.
>
>> Is this applicable only for standalone Mode?
>>
> It is not applicable for standalone mode, only for local.
>
>> Can I do this in a cluster setup, eg:
>> . setMaster("<hostname:port>[2]")..
>>
> No. It's faster to try than to ask a mailing list, actually. Also it's
> documented at
> http://spark.apache.org/docs/latest/submitting-applications.html#master-urls
> .
>
>> Is it number of threads per worker node?
>>
> You can control the number of total threads with
> spark-submit's --total-executor-cores parameter, if that's what you're
> looking for.
>
