Hi All, I have created a spark cluster on EC2 using spark-ec2 script. Whenever more data is there to be processed I would like to add new slaves to existing cluster and would like to remove slave node when the data to be processed is low.
It seems currently spark-ec2 doesn't have option to add/remove slaves to existing cluster. Could you please suggest how can we achieve this? One liner problem statement: How to add/remove slaves to/from Spark cluster on EC2 using spark-ec2? Suggestions on when to add/remove slaves is much appreciated. Thanks & Regards, Sirisha Devineni DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
