Thanks Jem, I do understand your suggestion. Actually --executor-cores alone doesn’t control the number of tasks, but is also governed by spark.task.cpus (amount of cores dedicated for each task’s execution).
Reframing my Question, How many threads can be spawned per executor core? Is it in user control? Regards, Sam From: Jem Tucker [mailto:[email protected]] Sent: Wednesday, August 26, 2015 2:26 PM To: Samya MAITI <[email protected]>; [email protected] Subject: Re: Relation between threads and executor core Hi Samya, When submitting an application with spark-submit the cores per executor can be set with --executor-cores, meaning you can run that many tasks per executor concurrently. The page below has some more details on submitting applications: https://spark.apache.org/docs/latest/submitting-applications.html thanks, Jem On Wed, Aug 26, 2015 at 9:47 AM Samya <[email protected]<mailto:[email protected]>> wrote: Hi All, Few basic queries :- 1. Is there a way we can control the number of threads per executor core? 2. Does this parameter “executor-cores” also has say in deciding how many threads to be run? Regards, Sam -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Relation-between-threads-and-executor-core-tp24456.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]>
