So I have a cluster in EC2 doing some work, and when I take a look here http://driver-node:4040/executors/
I see that my driver node is snoozing on the job: No tasks, no memory used, and no RDD blocks cached. I'm assuming that it was a conscious design choice not to have the driver node partake in the cluster's workload. Why is that? It seems like a wasted resource. What's more, the slaves may rise up one day and overthrow the driver out of resentment. Nick -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-doesn-t-the-driver-node-do-any-work-tp3909.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
