Re: Rolling upgrade Spark cluster

2014-12-17 Thread Bhaskar Dutta
HDFS rolling upgrade in Hadoop 2.6 (available since 2.4) http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html Some parts of NM and RM work preserving restart was released in Hadoop 2.6.0. YARN-1367 After restart NM should resync with the RM without killing

Rolling upgrade Spark cluster

2014-12-16 Thread Kenichi Maehashi
Hi, I have a Spark cluster using standalone mode. Spark Master is configured as High Availablity mode. Now I am going to upgrade Spark from 1.0 to 1.1, but don't want to interrupt the currently running jobs. (1) Are there any way to perform a rolling upgrade (while running a job)? (2) If not,