Thank you very much for replying and sorry for posting on the wrong list Best,
-- Nan Zhu School of Computer Science, McGill University On Thursday, October 24, 2013 at 1:06 AM, Jun Ping Du wrote: > Move to @user alias. > > ----- Original Message ----- > From: "Jun Ping Du" <j...@vmware.com (mailto:j...@vmware.com)> > To: gene...@hadoop.apache.org (mailto:gene...@hadoop.apache.org) > Sent: Wednesday, October 23, 2013 10:03:27 PM > Subject: Re: dynamically resizing Hadoop cluster on AWS? > > If only compute node (TaskTracker or NodeManager) in your instance, then > decommission nodes and shutdown related EC2 instances should be fine although > some finished/running tasks might need to be re-run automatically. If future, > we would support gracefully decommission (tracked by YARN-914 and > MAPREDUCE-5381) so that no tasks need to be rerun in this case (but need to > wait a while). > > Thanks, > > Junping > > ----- Original Message ----- > From: "Nan Zhu" <zhunans...@gmail.com (mailto:zhunans...@gmail.com)> > To: gene...@hadoop.apache.org (mailto:gene...@hadoop.apache.org) > Sent: Wednesday, October 23, 2013 8:15:51 PM > Subject: Re: dynamically resizing Hadoop cluster on AWS? > > Oh, I’m not running HDFS in the instances, I use S3 to save data > > -- > Nan Zhu > School of Computer Science, > McGill University > > > > On Wednesday, October 23, 2013 at 11:11 PM, Nan Zhu wrote: > > > Hi, all > > > > I’m running a Hadoop cluster on AWS EC2, > > > > I would like to dynamically resizing the cluster so as to reduce the cost, > > is there any solution to achieve this? > > > > E.g. I would like to cut the cluster size with a half, is it safe to just > > shutdown the instances (if some tasks are just running on them, can I rely > > on the speculative execution to re-run them on other nodes?) > > > > I cannot use EMR, since I’m running a customized version of Hadoop > > > > Best, > > > > -- > > Nan Zhu > > School of Computer Science, > > McGill University > > > > > > >
