How to add nodes

Hi, I am using instructions on http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ and http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/ to build a cluster. Currently my cluster has 4 nodes. Every time I add a node an

Re: Accessing Job counters displayed in WEB GUI in Hadoop Code

Guys ! How can i access Average map/reduce task run time for a job in Job Client Code ? Arun -- View this message in context: http://lucene.472066.n3.nabble.com/Accessing-Job-counters-displayed-in-WEB-GUI-in-Hadoop-Code-tp3576925p3578967.html Sent from the Hadoop lucene-users mailing list ar

RE: Grouping nodes into different racks in Hadoop Cluster

Hi ! i have three node cluster set up according to http://ankitasblogger.blogspot.com/2011/01/hadoop-cluster-setup.html http://ankitasblogger.blogspot.com/2011/01/hadoop-cluster-setup.html I have written a topology script and it doesn't work. For testing purpose i have also put simple script:

Re: Namenode does not start and generates no error messages

The problem was that my DFS system was not formatted. After running "hadoop namenode -format" I am able to launch the name node. This was confusing because I did have my DFS system formatted before. I must have deleted it and then forgotten that I deleted it. Thanks. On Sun, Dec 11, 2011 at 3:25

Awesome post on Hadoop. Some questions...

Really enthralled to read the post : http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/ Great job done. Some related questions: 1. The article says that hdfs always maintains 2 copies in the same rack and 3rd in a different rack. This only speeds up the hdfs "put" (

Re: ArrayWritable usage

Hi Dhruv, Is this working well for you ?? Are you able to do IntWritable [] abc = array.get(); I am trying similar thing for IntTwoDArrayWritable. The array.set works but array.get returns Writable[][] and I am not able to cast it to IntWritable[][]. -- View this message in context: http://lucen

Best ways to look-up information?

Hi, I am planning a system to process information with Hadoop, and I will have a few look-up tables that each processing node will need to query. There are perhaps 20-50 such tables, and each has on the order of one million entries. Which is the best mechanism for this look-up? Memcache, HBase, Ja

Re: Best ways to look-up information?

Mark, Are all of the tables used by all of the processes? Are all of the tables used all of the time or are some used infrequently? Does the data in these lookup tables change a lot or is it very stable? What is the actual size of the data, yes 1 million entries, but is this 1 million 1kB, 1

Jetty exception while running Hadoop

Hi, I keep getting the exception below. I've rebuild my EC2 cluster completely, and verified it on small jobs, but I still get it once I run anything sizable. The job runs, but I only get one part-0 file, even though I have 4 nodes and would expect for output files. Any help please? Thank you

Re: [ANNOUNCEMENT] Apache Hadoop 0.22.0 release

Great news! Is it alpha, beta or stable? Jie On Mon, Dec 12, 2011 at 3:33 PM, Konstantin Shvachko wrote: > On December 10, 2011 Hadoop PMC voted to release Hadoop 0.22.0 > See http://s.apache.org/COC > > The release have been brewing for one year. > It incorporates over 700 jiras fixed since the

Create a single output per each mapper

Hi, Suppose I have two mappers, each mapper is assigned 10 lines of data. I want to set a counter for each mapper, counting and accumulating, then output the counter value to the reducer when the mapper finishes processing all the assigned lines. So I want the mapper outputs values only when

More cores Vs More Nodes ?

Hey Guys, So I have a very naive question in my mind regarding Hadoop cluster nodes ? more cores or more nodes – Shall I spend money on going from 2-4 core machines, or spend money on buying more nodes less core eg. say 2 machines of 2 cores for example? Thanks, Praveenesh

Re: Create a single output per each mapper

Take a look at cleanup() method on Mapper. Thanks, Prashant Sent from my iPhone On Dec 12, 2011, at 8:46 PM, Shi Yu wrote: > Hi, > > Suppose I have two mappers, each mapper is assigned 10 lines of > data. I want to set a counter for each mapper, counting and > accumulating, then output the cou

Build failed when Wordcount example code has been changed

Hi guys ! I have set up a single node cluster as per below link http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#run-the-mapreduce-job I have tried to run Wordcount example. $bin/hadoop jar hadoop-*--examples.jar wordcount dfsinput dfsoutput It works. > I

Build failed when Wordcount example code has been changed

Hi guys ! I have set up a single node cluster as per below link http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#run-the-mapreduce-job I have tried to run Wordcount example. $bin/hadoop jar hadoop-*--examples.jar wordcount dfsinput dfsoutput It works. > I

Hadoop 0.22.0 Documentation: Config Files default point to diff(current) version Config

Hi All, I was going through Hadoop 0.22.0 Documentation at http://hadoop.apache.org/common/docs/r0.22.0/cluster_setup.html#Configuration+Files. The default config files (core-default, mapred-default, hdfs-default) are pointing to* CURRENT Hadoop release and not 0.22.0. * *For Example :* *core-def