Marcelo Vanzin wrote > I'm not sure I follow you here. This is something that you are > defining, not Spark.
Yes, you are right. In my code, 1) my notion of RUNNING is that both driver + executors are in RUNNING state. 2) my notion of WAITING is if any one of driver/executor is in WAITING state. So, - SparkLauncher provides me the details about the "driver". RUNNING/SUBMITTED/WAITING - SparkListener provides me the details about the "executor" using onExecutorAdded/onExecutorDeleted I want to combine both SparkLauncher + SparkListener to achieve my view of RUNNING/WAITING. The only thing confusing me here is that I don't know how Spark internally converts applications from WAITING to RUNNING state. For example, if an application wanted 4 executors (spark.executor.instances=4) but the spark cluster can only provide 1 executor. This means that I will only receive 1 onExecutorAdded event. Will the application state change to RUNNING (even if 1 executor was allocated)? If I am clear on this logic I can implement my feature. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
