Mohit,
New datanodes will connect to the namenode so thats how the namenode
knows. Just make sure the datanodes have the correct {fs.default.dir}
in their hdfs-site.xml and then start them. The namenode can, however,
choose to reject the datanode if you are using the {dfs.hosts} and
{dfs.ho
Thanks all for the answers!!
On Thu, Mar 1, 2012 at 5:52 PM, Arpit Gupta wrote:
> It is initiated by the slave.
>
> If you have defined files to state which slaves can talk to the namenode
> (using config dfs.hosts) and which hosts cannot (using
> property dfs.hosts.exclude) then you would need
It is initiated by the slave. If you have defined files to state which slaves can talk to the namenode (using config dfs.hosts) and which hosts cannot (using property dfs.hosts.exclude) then you would need to edit these files and issue the refresh command.On Mar 1, 2012, at 5:35 PM, Mohit Anchlia w
pache.org; Raj Vishwanathan
>Sent: Thursday, March 1, 2012 5:42 PM
>Subject: Re: Adding nodes
>
>Whatever Joey said is correct for Cloudera's distribution. For same, I am
>not confident about other distribution as i haven't tried them.
>
>Thanks,
>Anil
>
>
the
> correct daemons on the correct nodes from the master node.
>
>
> Raj
>
>
> >
> > From: Joey Echeverria
> >To: "common-user@hadoop.apache.org"
> >Cc: "common-user@hadoop.apache.org"
> >Se
On Thu, Mar 1, 2012 at 4:57 PM, Joey Echeverria wrote:
> Not quite. Datanodes get the namenode host from fs.defalt.name in
> core-site.xml. Task trackers find the job tracker from the
> mapred.job.tracker setting in mapred-site.xml.
>
I actually meant to ask how does namenode/jobtracker know the
ache.org"
>Sent: Thursday, March 1, 2012 4:57 PM
>Subject: Re: Adding nodes
>
>Not quite. Datanodes get the namenode host from fs.defalt.name in
>core-site.xml. Task trackers find the job tracker from the mapred.job.tracker
>setting in mapred-site.xml.
>
>Sent from my
Not quite. Datanodes get the namenode host from fs.defalt.name in
core-site.xml. Task trackers find the job tracker from the mapred.job.tracker
setting in mapred-site.xml.
Sent from my iPhone
On Mar 1, 2012, at 18:49, Mohit Anchlia wrote:
> On Thu, Mar 1, 2012 at 4:46 PM, Joey Echeverria wr
On Thu, Mar 1, 2012 at 4:46 PM, Joey Echeverria wrote:
> You only have to refresh nodes if you're making use of an allows file.
>
> Thanks does it mean that when tasktracker/datanode starts up it
communicates with namenode using master file?
Sent from my iPhone
>
> On Mar 1, 2012, at 18:29, Mohi
You only have to refresh nodes if you're making use of an allows file.
Sent from my iPhone
On Mar 1, 2012, at 18:29, Mohit Anchlia wrote:
> Is this the right procedure to add nodes? I took some from hadoop wiki FAQ:
>
> http://wiki.apache.org/hadoop/FAQ
>
> 1. Update conf/slave
> 2. on the s
Is this the right procedure to add nodes? I took some from hadoop wiki FAQ:
http://wiki.apache.org/hadoop/FAQ
1. Update conf/slave
2. on the slave nodes start datanode and tasktracker
3. hadoop balancer
Do I also need to run dfsadmin -refreshnodes?
Thanks for all the input. I am trying to do cluster setup in EC2 but not
able to find how i can do dns updation centrally. If anyone one knows how
to do this please help me ..
On Sat, Dec 17, 2011 at 8:10 PM, Michel Segel wrote:
> Actually I would recommend avoiding /etc/hosts and using DNS if th
Actually I would recommend avoiding /etc/hosts and using DNS if this is going
to be a production grade cluster...
Sent from a remote device. Please excuse any typos...
Mike Segel
On Dec 17, 2011, at 5:40 AM, alo alt wrote:
> Hi,
>
> in the slave - file too. /etc/hosts is also recommend to av
Hi,
in the slave - file too. /etc/hosts is also recommend to avoid DNS
issues. After adding in slaves the new node has to be started and
should quickly appear in the web-ui. If you don't need the nodes all
time you can setup a exclude and refresh your cluster
(http://wiki.apache.org/hadoop/FAQ#I_w
Madhu,
On Sat, Dec 17, 2011 at 4:36 PM, madhu phatak wrote:
> When i add a node to running cluster do i have to add its hostname to all
> nodes (slaves +master) /etc/hosts file?
Yes.
> Or some other way is there?
You can run a DNS, and have the resolution centrally managed.
--
Harsh J
Hi,
I am trying to add nodes dynamically to a running hadoop cluster.I started
tasktracker and datanode in the node. It works fine. But when some node
try fetch values ( for reduce phase) it fails with unknown host exception.
When i add a node to running cluster do i have to add its hostname to al
Hi all,
We are currently in the process of replacing the servers in our Hadoop
0.20.2 production cluster and in the last couple of days have
experienced an error similar to the following (from the JobTracker
log) several times, which then appears to hang the JobTracker:
2010-10-15 04:13:38,980 IN
17 matches
Mail list logo
