Hi, I have read in many materials (including from the book: Spark - The Definitive Guide) that Spark is a compiler.
In my understanding, our program is used until the point of DAG generation. This portion can be written in any language - Java,Scala,R,Python. Post that (executing the DAG), the engine runs in Scala only. This leads to Spark being called as a compiler. If the above is true, we need to install R / Python only in the driver machine. R / Python run time is not needed in worker nodes. Am I correct ? Thanks!
