Hey Reza,
From reading your code, you are calling this for the key "sex":
output.collect("The total population is: ", (actual population))
and, for every other key:
output.collect("The total population is: ", 0)
You probably only want to call the output collector in the first case,
not every time.
Brian
On Apr 18, 2009, at 12:21 PM, Reza wrote:
Hello all:
I am new to Hadoop and Map Reduce. I am writing a program to analyze
some
census data.
I have a general question with MapReduce:
In the Reducer, how can I separate keys to do separate calculations
based
on the key? In my case, I am trying to use if statements to separate
the
keys out, but for some reason, it is not doing so.
Here is a segment of my code. My mapper seems to work, I think it is
my
Reducer that is messing up:
public static class MapClass extends MapReduceBase implements
Mapper<LongWritable, Text, Text, IntWritable> {
// private final static IntWritable one = new IntWritable(1);
Text sex = new Text("sex");
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer itr = new StringTokenizer(line,",");
:
:
word.set(itr.nextToken());
IntWritable sexVal = new
IntWritable(Integer.parseInt(word.toString())); //map gender
output.collect(sex, sexVal);
}//end map()method
}//end class MapClass
public static class Reduce extends MapReduceBase
implements Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterator<IntWritable> values,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
int totalPop = 0; //count: total population
int numMales = 0; //count: males
int numFemales = 0; //count: females
Text popOut = new Text("The total population is:
");
if (key.equals("sex")) {
while (values.hasNext()) {
if (values.next().get() == 0)
numMales++; //number
of males
else
numFemales++; //number
of
females
}//end while
totalPop = numMales + numFemales; //count of population
}//end if
output.collect(popOut, new IntWritable(totalPop));
}//end reduce method
}//end class Reduce
key is of type Text, I also used: if (key.toString() == "sex) {....
but that wasnt working either.
There are of course other keys and values, but I would think to
gather all
the keys "sex" would be done with the way I show it above. However,
My
output reads as 0. I tried changing the numMales value to see if
the if
statement is executed, but it is not. Am I doing something wrong?
