RE: The number of cores vs. the number of executors

2014-07-07 Thread innowireless TaeYun Kim
For your information, I've attached the Ganglia monitoring screen capture on the Stack Overflow question. Please see: http://stackoverflow.com/questions/24622108/apache-spark-the-number-of-cores -vs-the-number-of-executors From: innowireless TaeYun Kim [mailto:taeyun@innowireless.

The number of cores vs. the number of executors

2014-07-07 Thread innowireless TaeYun Kim
Hi, I'm trying to understand the relationship of the number of cores and the number of executors when running a Spark job on YARN. The test environment is as follows: - # of data nodes: 3 - Data node machine spec: - CPU: Core i7-4790 (# of cores: 4, # of threads: 8) - RAM: 32GB (8GB