It should. BTW what's the problem with decommissioning?It's the correct way of removing the DN, as said by Charles earlier. Create the exclude file and add the property in the hdfs-site.xml file.
Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Thu, Feb 14, 2013 at 5:05 AM, Arko Provo Mukherjee < arkoprovomukher...@gmail.com> wrote: > Hi, > > I see that the NameNode IP is present in conf/slaves file. > > Will removing the IP of the NN from the conf/slaves file and > restarting hadoop help? > > Thanks & regards > Arko > > On Wed, Feb 13, 2013 at 5:32 PM, Arko Provo Mukherjee > <arkoprovomukher...@gmail.com> wrote: > > Hi, > > > > I have 10 nodes. 1 Master Node (running all deamons) and 9 Slave Nodes > > (running DN and Task Tracker). > > > > My issue is that the my NN is getting filled up pretty fast as the NN > > acts as a DN. > > > > I want my NN not to function as the DN. > > > > Does including the IP for the NN in the "dfs.hosts.exclude" and > > running "$ hadoop dfsadmin -refreshNodes" solve my problem? > > > > Thanks & regards > > Arko > > > > On Wed, Feb 13, 2013 at 5:22 PM, Mohammad Tariq <donta...@gmail.com> > wrote: > >> Stopping DN on one machine means excluding it. What exactly is the > >> requirement? How many nodes do you have right now? > >> > >> Warm Regards, > >> Tariq > >> https://mtariq.jux.com/ > >> cloudfront.blogspot.com > >> > >> > >> On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee > >> <arkoprovomukher...@gmail.com> wrote: > >>> > >>> Hi, > >>> > >>> I am not sure the provided link matches my situation. > >>> > >>> That is because, I am not trying to exclude any node but just trying > >>> to stop the NN act as a DataNode. > >>> > >>> Thanks & regards > >>> Arko > >>> > >>> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee > >>> <arkoprovomukher...@gmail.com> wrote: > >>> > Thanks! Let me study and find out if it works for me. > >>> > Thanks & regards > >>> > Arko > >>> > > >>> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee > >>> > <arkoprovomukher...@gmail.com> wrote: > >>> >> Hi, > >>> >> > >>> >> But the new location will also be the same NN physical server. > >>> >> So how does that help? > >>> >> > >>> >> I would like to clear up the space in the master by removing all the > >>> >> HDFS data to the slave nodes. > >>> >> > >>> >> Thanks & regards > >>> >> Arko > >>> >> > >>> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <donta...@gmail.com > > > >>> >> wrote: > >>> >>> No, copy this directory to the new location of your choice. > >>> >>> > >>> >>> Warm Regards, > >>> >>> Tariq > >>> >>> https://mtariq.jux.com/ > >>> >>> cloudfront.blogspot.com > >>> >>> > >>> >>> > >>> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee > >>> >>> <arkoprovomukher...@gmail.com> wrote: > >>> >>>> > >>> >>>> Thanks! > >>> >>>> However, in that case, how do I handle the current data that is > >>> >>>> already in > >>> >>>> NN? > >>> >>>> Do I have to reformat HDFS? > >>> >>>> Thanks & regards > >>> >>>> Arko > >>> >>>> > >>> >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq < > donta...@gmail.com> > >>> >>>> wrote: > >>> >>>> > Just don't run the DN daemon on that machine. Dedicate that > machine > >>> >>>> > to > >>> >>>> > NN. > >>> >>>> > Remove the hostname of the NN machine from the 'slaves' file. > >>> >>>> > > >>> >>>> > Warm Regards, > >>> >>>> > Tariq > >>> >>>> > https://mtariq.jux.com/ > >>> >>>> > cloudfront.blogspot.com > >>> >>>> > > >>> >>>> > > >>> >>>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee > >>> >>>> > <arkoprovomukher...@gmail.com> wrote: > >>> >>>> >> > >>> >>>> >> Hi, > >>> >>>> >> > >>> >>>> >> Thanks for the help! > >>> >>>> >> > >>> >>>> >> However, I am still unsure about how to "turn off" the datanode > >>> >>>> >> feature on the NN. > >>> >>>> >> > >>> >>>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is > pointed > >>> >>>> >> to > >>> >>>> >> a directory. > >>> >>>> >> > >>> >>>> >> Should I just comment out that property? What would happen to > the > >>> >>>> >> current data that there is in the Master? Will it get removed > >>> >>>> >> automatically? > >>> >>>> >> > >>> >>>> >> Thanks & regards > >>> >>>> >> Arko > >>> >>>> >> > >>> >>>> >> > >>> >>>> >> > >>> >>>> >> On Wed, Feb 13, 2013 at 1:55 PM, Mohammad Tariq > >>> >>>> >> <donta...@gmail.com> > >>> >>>> >> wrote: > >>> >>>> >> > You can specify the logging level as specified by Charles. > But > >>> >>>> >> > turning > >>> >>>> >> > logs > >>> >>>> >> > off is never a good idea. Logs are really helpful in problem > >>> >>>> >> > diagnosis, > >>> >>>> >> > which are eventual. > >>> >>>> >> > > >>> >>>> >> > Warm Regards, > >>> >>>> >> > Tariq > >>> >>>> >> > https://mtariq.jux.com/ > >>> >>>> >> > cloudfront.blogspot.com > >>> >>>> >> > > >>> >>>> >> > > >>> >>>> >> > On Thu, Feb 14, 2013 at 1:22 AM, Arko Provo Mukherjee > >>> >>>> >> > <arkoprovomukher...@gmail.com> wrote: > >>> >>>> >> >> > >>> >>>> >> >> Hi, > >>> >>>> >> >> > >>> >>>> >> >> Yeah, my NameNode is also seconding as a DataNode. > >>> >>>> >> >> > >>> >>>> >> >> I would like to "turn off" this feature. > >>> >>>> >> >> > >>> >>>> >> >> Request help regarding the same. > >>> >>>> >> >> > >>> >>>> >> >> Thanks & regards > >>> >>>> >> >> Arko > >>> >>>> >> >> > >>> >>>> >> >> On Wed, Feb 13, 2013 at 1:38 PM, Charles Baker < > cba...@sdl.com> > >>> >>>> >> >> wrote: > >>> >>>> >> >> > Hi Arko. Sounds like you may be running a DataNode on the > >>> >>>> >> >> > NameNode > >>> >>>> >> >> > which > >>> >>>> >> >> > is > >>> >>>> >> >> > not recommended practice. Normally, the only files the NN > >>> >>>> >> >> > stores > >>> >>>> >> >> > are > >>> >>>> >> >> > the > >>> >>>> >> >> > image and edits files. It does not store any actual HDFS > >>> >>>> >> >> > data. If > >>> >>>> >> >> > you > >>> >>>> >> >> > must > >>> >>>> >> >> > run a DN on the NN, try turning down the logging in > >>> >>>> >> >> > /conf/log4j.properties: > >>> >>>> >> >> > > >>> >>>> >> >> > #hadoop.root.logger=INFO,console > >>> >>>> >> >> > #hadoop.root.logger=WARN,console > >>> >>>> >> >> > hadoop.root.logger=ERROR,console > >>> >>>> >> >> > > >>> >>>> >> >> > Depending on the logging information you require, of > course. > >>> >>>> >> >> > > >>> >>>> >> >> > -Chuck > >>> >>>> >> >> > > >>> >>>> >> >> > > >>> >>>> >> >> > -----Original Message----- > >>> >>>> >> >> > From: Arko Provo Mukherjee > >>> >>>> >> >> > [mailto:arkoprovomukher...@gmail.com] > >>> >>>> >> >> > Sent: Wednesday, February 13, 2013 11:32 AM > >>> >>>> >> >> > To: hdfs-user@hadoop.apache.org > >>> >>>> >> >> > Subject: Managing space in Master Node > >>> >>>> >> >> > > >>> >>>> >> >> > Hello Gurus, > >>> >>>> >> >> > > >>> >>>> >> >> > I am managing a Hadoop Cluster to run some experiments. > >>> >>>> >> >> > > >>> >>>> >> >> > The issue I am continuously facing is that the Master Node > >>> >>>> >> >> > runs > >>> >>>> >> >> > out > >>> >>>> >> >> > of > >>> >>>> >> >> > disk > >>> >>>> >> >> > space due to logs and data files. > >>> >>>> >> >> > > >>> >>>> >> >> > I can monitor and delete log files. However, I cannot > delete > >>> >>>> >> >> > the > >>> >>>> >> >> > HDFS > >>> >>>> >> >> > data. > >>> >>>> >> >> > > >>> >>>> >> >> > Thus, is there a way to force Hadoop not to save any HDFS > >>> >>>> >> >> > data in > >>> >>>> >> >> > the > >>> >>>> >> >> > Master > >>> >>>> >> >> > Node? > >>> >>>> >> >> > > >>> >>>> >> >> > Then I can use my master to handle the metadata only and > >>> >>>> >> >> > store the > >>> >>>> >> >> > logs. > >>> >>>> >> >> > > >>> >>>> >> >> > Thanks & regards > >>> >>>> >> >> > Arko > >>> >>>> >> >> > SDL Enterprise Technologies, Inc. - all rights reserved. > The > >>> >>>> >> >> > information contained in this email may be confidential > >>> >>>> >> >> > and/or > >>> >>>> >> >> > legally > >>> >>>> >> >> > privileged. It has been sent for the sole use of the > intended > >>> >>>> >> >> > recipient(s). > >>> >>>> >> >> > If you are not the intended recipient of this mail, you > are > >>> >>>> >> >> > hereby > >>> >>>> >> >> > notified > >>> >>>> >> >> > that any unauthorized review, use, disclosure, > dissemination, > >>> >>>> >> >> > distribution, > >>> >>>> >> >> > or copying of this communication, or any of its contents, > is > >>> >>>> >> >> > strictly > >>> >>>> >> >> > prohibited. If you have received this communication in > error, > >>> >>>> >> >> > please > >>> >>>> >> >> > reply > >>> >>>> >> >> > to the sender and destroy all copies of the message. > >>> >>>> >> >> > Registered address: 201 Edgewater Drive, Suite 225, > >>> >>>> >> >> > Wakefield, MA > >>> >>>> >> >> > 01880, > >>> >>>> >> >> > USA > >>> >>>> >> >> > > >>> >>>> >> > > >>> >>>> >> > > >>> >>>> > > >>> >>>> > > >>> >>> > >>> >>> > >> > >> >
