Hi,
In general YARN is used as the resource scheduler regardless of the execution engine whether it is MapReduce or Spark. Yarn will create a resource container for the submitted job (that is the Spark client) and will execute it in the default engine (in this case Spark). There will be a job scheduler and one or more Spark Executors depending on the cluster. So as far as I can see both diagrams are correct, HTH Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15 <http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7. co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4 Publications due shortly: Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8 Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: Nisrina Luthfiyati [mailto:nisrina.luthfiy...@gmail.com] Sent: 27 November 2015 11:12 To: user@spark.apache.org Subject: In yarn-client mode, is it the driver or application master that issue commands to executors? Hi all, I'm trying to understand how yarn-client mode works and found these two diagrams: <http://image.slidesharecdn.com/th-1150a-hall1-feng-v2-140617142634-phpapp01/95/sparkonyarn-empower-spark-applications-on-hadoop-cluster-10-638.jpg?cb=1403015417> <http://blog.cloudera.com/wp-content/uploads/2014/05/spark-yarn-f22.png> In the first diagram, it looks like the driver running in client directly communicates with executors to issue application commands, while in the second diagram it looks like application commands is sent to application master first and then forwarded to executors. Would anyone knows which case is true or is there any other interpretation to these diagrams? Thanks! Nisrina
