Re: Dynamic resource allocation in Standalone mode

2015-07-20 Thread Andrew Or
Hi Ray, In standalone mode, you have this thing called the SparkDeploySchedulerBackend, which has this thing called the AppClient. This is the thing on the driver side that already talks to the Master to register the application. As for dynamic allocation in standalone mode, I literally *just*

Dynamic resource allocation in Standalone mode

2015-07-18 Thread Dogtail Ray
Hi all, I am planning to dynamically increase or decrease the number of executors allocated to an application during runtime, and it is similar to dynamic resource allocation, which is only feasible in Spark on Yarn mode. Any suggestions on how to implement this feature in Standalone mode? My