Add new Datnodes : Is redistribution of previous data required?

2009-06-24 Thread asif md
hello everyone, I have added 7 nodes to my 3 node cluster. I followed the following steps to do this 1. added the node's ip to conf/slaves at master 2. ran bin/start-balance.sh at each node As i loaded the data when the size of the cluster was three which is now TEN. Can i do anything to

Re: Add new Datnodes : Is redistribution of previous data required?

2009-06-24 Thread asif md
of your data for you. That is, after you've run the rebalancer, your data should be stored evenly among your 10 nodes. Alex On Wed, Jun 24, 2009 at 2:50 PM, asif md asif.d...@gmail.com wrote: hello everyone, I have added 7 nodes to my 3 node cluster. I followed the following steps to do

Re: Add new Datnodes : Is redistribution of previous data required?

2009-06-24 Thread asif md
/core/docs/current/hdfs_user_guide.html#Rebalancer http://hadoop.apache.org/core/docs/current/commands_manual.html#balancer http://issues.apache.org/jira/secure/attachment/12368261/RebalanceDesign6.pdf --Konstantin asif md wrote: @Alex Thanks. http://wiki.apache.org/hadoop/FAQ#6

ReduceTask: No Route To Host

2009-06-06 Thread asif md
Now theres this error showing up. When i run a job on my 2 node cluster, it hangs at [ ~]$ hadoop jar $HADOOP_HOME/hadoop-0.18.3-examples.jar wordcount gutenberg gutenberg-output 09/06/06 01:50:54 INFO mapred.FileInputFormat: Total input paths to process : 6 09/06/06 01:50:54 INFO

No Route To Host at Slave

2009-06-05 Thread asif md
Hello all, I'm strugling to fix the 'NO ROUTE TO HOST' problem on my Only Slave. The datanode log is as follows 2009-06-05 15:12:41,076 INFO org.apache.hadoop.dfs.DataNode: STARTUP_MSG: / STARTUP_MSG: Starting DataNode STARTUP_MSG:

Re: No Route To Host at Slave

2009-06-05 Thread asif md
and Regards Asif. On Fri, Jun 5, 2009 at 5:24 PM, asif md asif.d...@gmail.com wrote: Hello all, I'm strugling to fix the 'NO ROUTE TO HOST' problem on my Only Slave. The datanode log is as follows 2009-06-05 15:12:41,076 INFO org.apache.hadoop.dfs.DataNode: STARTUP_MSG

Re: No Route To Host at Slave

2009-06-05 Thread asif md
) scanned in 0.094 seconds On Fri, Jun 5, 2009 at 7:13 PM, asif md asif.d...@gmail.com wrote: I've found the problem but am clueless about how to fix it. wen i did the following on the master after running $HADOOP_HOME/bin/start-dfs.sh [ ~]$ nmap -PN -p54310 localhost Starting Nmap 4.11

Cluster Setup Issues : Datanode not being initialized.

2009-06-04 Thread asif md
Hello all, I'm trying to setup a two node cluster remote using the following tutorials { NOTE : i'm ignoring the tmp directory property in hadoop-site.xml suggested by Michael } Running Hadoop On Ubuntu Linux (Single-Node Cluster) - Michael G.

Re: Cluster Setup Issues : Datanode not being initialized.

2009-06-04 Thread asif md
: / SHUTDOWN_MSG: Shutting down DataNode at *** **88 Please suggest. Asif. On Thu, Jun 4, 2009 at 4:15 PM, asif md asif.d...@gmail.com wrote: @Ravi thanx ravi

Re: Cluster Setup Issues : Datanode not being initialized.

2009-06-04 Thread asif md
@ Ravi. Not able to do that. On Thu, Jun 4, 2009 at 5:38 PM, Raghu Angadi rang...@yahoo-inc.com wrote: Did you try 'telnet 198.55.35.229 54310' from this datanode? The log show that it is not able to connect to master:54310. ssh from datanode does not matter. Raghu. asif md wrote: I

Re: PIG and Hive

2009-05-06 Thread asif md
http://www.cloudera.com/hadoop-training-hive-introduction http://www.cloudera.com/hadoop-training-pig-introduction On Wed, May 6, 2009 at 1:17 AM, Ricky Ho r...@adobe.com wrote: Are they competing technologies of providing a higher level language for Map/Reduce programming ? Or are they

Chaining Multiple Map reduce jobs.

2009-04-08 Thread asif md
hi everyone, i have to chain multiple map reduce jobs actually 2 to 4 jobs , each of the jobs depends on the o/p of preceding job. In the reducer of each job I'm doing very little just grouping by key from the maps. I want to give the output of one MapReduce job to the next job without having