Re: Is it possible to write each key-value pair emitted by the reducer to a different output file

Hi Amareshwari, But this feature is not available in Hadoop 0.18.3. Is there any work around for this version. Thanks, Udaya. On Fri, Feb 5, 2010 at 10:49 AM, Amareshwari Sri Ramadasu < amar...@yahoo-inc.com> wrote: > See MultipleOutputs at > http://hadoop.apache.org/common/docs/r0.20.0/api/o

Re: Is there Synthetic Load generator for Hadoop-0.18.3

On Sat, Feb 6, 2010 at 12:07 AM, Todd Lipcon wrote: > Why not try downloading the load gen source from 0.19 or 0.20, > compiling against 0.18, and seeing if there are any errors? If there > are, fix them. The APIs haven't changed much in a long time, it > shouldn't be a lot of work. And, for hdfs

Re: Datanode will shut down automatically after it starts

On Fri, Feb 5, 2010 at 11:24 PM, gillggx wrote: > 2010-02-05 21:53:00,644 ERROR > org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call > to /192.168.0.1:9000 failed on local exception: java.io.IOException: > Connection reset by peer you'd better check the log of namenode 192

Re: Not able to compile '.java' files

Thanks for the help, Brien. --Prateek. On Fri, Feb 5, 2010 at 11:26 PM, brien colwell wrote: > To get a feel for Hadoop, I'd recommend using Eclipse and using a single > node to start. If you add all the Hadoop JARs to your Eclipse build path, I > think there are five, then Eclipse will manage

Re: Not able to compile '.java' files

Hi, > 1) I have 3 files, namely MaxTemperature.java, MaxTemperatureMapper.java, > MaxTemperatureReducer.java. Now, I have to compile them to get the '.class' > files which would be used by 'hadoop' command. I tried the following: > > 'javac -cp .:/hadoop/lib MaxTemperatureMapper.java' > > But it g

Re: Not able to compile '.java' files

Hi Mikhail, Thanks a lot for your clear response. --Prateek. On Sat, Feb 6, 2010 at 4:06 PM, Mikhail Yakshin wrote: > Hi, > > > 1) I have 3 files, namely MaxTemperature.java, MaxTemperatureMapper.java, > > MaxTemperatureReducer.java. Now, I have to compile them to get the > '.class' > > files w

Re: Is it possible to write each key-value pair emitted by the reducer to a different output file

It's possible to write your own class to take better encapsulate writing of side-effect files, but as people have said, you can run into unanticipated issues if the number of files you try to write at once becomes high. On Sat, Feb 6, 2010 at 3:47 AM, Udaya Lakshmi wrote: > Hi Amareshwari, > >

has anyone ported hadoop.lib.aggregate?

>From what I can tell, while the ValueAggregator stuff should be useable, the ValueAggregatorJob and ValueAggregatorJobBase classes still use the old Mapper and Reducer signatures, and basically aren't compatible with the new mapreduce.* API. Is that correct? Has anyone out there done a port? We'v

Duplicate keys on reduce output

Hi All, I am working Amazon MR since last 2 weeks. I created Hadoop Map Reduce Program from release 0.18.3 which Amazon Ec2 supports. The issue is that map output collector only sends one key , but when I iterate in reduce method I get duplicate keys. Although locally it works fine but on EC2 i