Hi all, I am planning to dynamically increase or decrease the number of executors allocated to an application during runtime, and it is similar to dynamic resource allocation, which is only feasible in Spark on Yarn mode. Any suggestions on how to implement this feature in Standalone mode?
My current problem is: I want to send a ADD_EXECUTOR command from scheduler module (in CoarseGrainedSchedulerBackend.scala) to deploy module (in Master.scala), but don't know how to communicate between the two modules.... Great thanks for any suggestions!
