[ANN] Multireducers - run multiple reducers on the same mapreduce job

I'll appreciate reviews of the code and the API of multireducers - a way to run a couple of map and reduce classes in the same MapReduce job. Thanks, https://github.com/elazarl/multireducers Usage example: MultiJob.create(). withMapper(SelectFirstField.class, Text.class, IntWritable.cla

Re: hadoop security

The reason you must use jsvc, is to start as root and then bind to privilleged ports. Secure Datanode won't run on non privilleged port (i.e. >1024). The advantages are, users on your machine cannot take this port and pretend they're DN. Only root can. Like you, I don't think this should be manda

[ANN] hadoopconf - Hadoop configuration helper tool

lpers#readme I'd love to receive feedback, and to understand if indeed this utility is useful to the community. For example, will hbase support be useful? Feel free to send me bug reports, feature requests, notes and suggestions. *Be warned*, it is still a very new code in alpha stage. Than

Re: Documentation for Hadoop's RPC mechanism

Done https://issues.apache.org/jira/browse/HADOOP-9892 On Tue, Aug 20, 2013 at 9:01 PM, Suresh Srinivas wrote: > Create a Jira and post it into hadoop documentation. I can help you with > the review and commit. > > Sent from phone > > On Aug 20, 2013, at 10:40 AM, Elaza

Documentation for Hadoop's RPC mechanism

Hi, I've written some documentationfor Hadoop's RPC mechanism internals: http://hadoop.quora.com/Hadoop-RPC-mechanism I'll be very happy if the community can review it. You should be able to edit it directly, or just send your comments to the list.

Re: Why Hadoop force using DNS?

reply to this will be very > appreciated. > > From: Elazar Leibovich > Reply-To: "user@hadoop.apache.org" > Date: Thursday, July 25, 2013 3:51 AM > To: user > Subject: Why Hadoop force using DNS? > > Looking at Hadoop source you can see that Hadoop relies on

Re: Why Hadoop force using DNS?

Why not remove it? >> >> Greg Bledsoe >> >> From: 武泽胜 >> Reply-To: "user@hadoop.apache.org" >> Date: Mon, 29 Jul 2013 08:21:51 -0500 >> To: "user@hadoop.apache.org" >> Subject: Re: Why Hadoop force using DNS? >> >> I h

Why Hadoop force using DNS?

Looking at Hadoop source you can see that Hadoop relies on the fact each node has resolvable name. For example, Hadoop 2 namenode reverse look the up of each node that connects to it. Also, there's no way way to tell a database to advertise an UP as it's address. Setting datanode.network.interface

Saving counters in Mapfile

Hi, A common use case one want an ordered structure for, is for saving counters. Naturally, I wanted to save the counters in a Mapfile: for (long ix = 0; ix < MAXVALUE; ix++) { mapfile.append(new Text("counter key of val " + ix), new LongWritable(ix)); } This however looks a bit