Re: Multiple Input Data Processing using MapReduce

2010-12-30 Thread Michael Toback
I am faced with a similar problem. I want to process an entire set of bugs including their entire history. Once. Then, incrementally process a combination of the latest output + the changes since last processed. I hit upon a way of handling multiple outputs. Perhaps if there was something in the

Re: how to build hadoop in Linux

2010-12-30 Thread Konstantin Boudnik
The Java5 dependency is about to go from Hadoop. See HADOOP-7072. I will try to commit it first thing next year. So, wait a couple of days and you'll be all right. Happy New Year everyone! On Thu, Dec 30, 2010 at 22:08, Da Zheng wrote: > Hello, > > I need to build hadoop in Linux as I need to m

how to build hadoop in Linux

2010-12-30 Thread Da Zheng
Hello, I need to build hadoop in Linux as I need to make some small changes in the code, but I don't know what is the simplest way to build hadoop. I googled it and so far I only found two places that tell how to build hadoop. One is http://bigdata.wordpress.com/2010/05/27/hadoop-cookbook-3-how-to

Re: Flow of control

2010-12-30 Thread maha
Very helpful :) thanks Ping. Maha On Dec 30, 2010, at 6:13 PM, li ping wrote: > On Fri, Dec 31, 2010 at 9:28 AM, maha wrote: > >> Hi, >> >> (1) I declared a global variable in my hadoop mainClass which gets >> initialized in the 'run' function of this mainClass. When I try to access >> this g

Re: Retrying connect to server

2010-12-30 Thread James Seigel
Or 3) The configuration (or lack thereof) on the machine you are trying to run this, has no idea where your DFS or JobTracker is :) Cheers James. On 2010-12-30, at 8:53 PM, Adarsh Sharma wrote: > Cavus,M.,Fa. Post Direkt wrote: >> I process this >> >> ./hadoop jar ../../hadoopjar/

Re: Retrying connect to server

2010-12-30 Thread Adarsh Sharma
Cavus,M.,Fa. Post Direkt wrote: I process this ./hadoop jar ../../hadoopjar/hd.jar org.postdirekt.hadoop.WordCount gutenberg gutenberg-output I get this Dıd anyone know why I get this Error? 10/12/30 16:48:59 INFO security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixG

Re: Flow of control

2010-12-30 Thread li ping
On Fri, Dec 31, 2010 at 9:28 AM, maha wrote: > Hi, > > (1) I declared a global variable in my hadoop mainClass which gets > initialized in the 'run' function of this mainClass. When I try to access > this global static variable from the MapperClass, it appears to be > uninitialized. > >W

Re: Retrying connect to server

2010-12-30 Thread li ping
make sure your /etc/hosts file contains the correct ip/hostname pair. This is very important 2010/12/30 Cavus,M.,Fa. Post Direkt > I process this > > ./hadoop jar ../../hadoopjar/hd.jar org.postdirekt.hadoop.WordCount > gutenberg gutenberg-output > > I get this > Dıd anyone know why I get this E

Flow of control

2010-12-30 Thread maha
Hi, (1) I declared a global variable in my hadoop mainClass which gets initialized in the 'run' function of this mainClass. When I try to access this global static variable from the MapperClass, it appears to be uninitialized. Why is that? Is it because of the parallel execution of H

Re: Retrying connect to server

2010-12-30 Thread maha
Hi Cavus, Please check that hadoop JobTracker and other daemons are running by typing "jps". If you see one of (JobTracker,TaskTracker,namenode,datanode) missing then you need to 'stop-all' then format the namenode and start-all again. Maha On Dec 30, 2010, at 7:52 AM, Cavus,M.,Fa. Post D

Re: Retrying connect to server

2010-12-30 Thread Esteban Gutierrez Moguel
Hello Cavus, is your Job Tracker running on localhost? It would be great if you can provide more information about your current Hadoop setup. cheers, esteban. estebangutierrez.com — twitter.com/esteban 2010/12/30 Cavus,M.,Fa. Post Direkt > I process this > > ./hadoop jar ../../hadoopjar/hd.

Retrying connect to server

2010-12-30 Thread Cavus,M.,Fa. Post Direkt
I process this ./hadoop jar ../../hadoopjar/hd.jar org.postdirekt.hadoop.WordCount gutenberg gutenberg-output I get this Dıd anyone know why I get this Error? 10/12/30 16:48:59 INFO security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=30

Re: documentation of hadoop implementation

2010-12-30 Thread Mark Kerzner
Thanks, Da, this makes you a better Googler, and an expert one. Cheers, Mark On Thu, Dec 30, 2010 at 9:25 AM, Da Zheng wrote: > there is someone else like me who had problems to find it:-) I thought I > was the > only one who had the problem, so I didn't send the link. > > http://developer.yaho

Re: documentation of hadoop implementation

2010-12-30 Thread Da Zheng
there is someone else like me who had problems to find it:-) I thought I was the only one who had the problem, so I didn't send the link. http://developer.yahoo.com/blogs/hadoop/posts/2010/01/hadoop_bay_area_january_2010_u/ Best, Da On 12/30/10 12:24 AM, Mark Kerzner wrote: > Da, where did you fi