Indeed, worked like a charm.
Some strange thing that I see is that the more reducers I run, the more
data I get on the output. However, my suspicion is that since I use some
global counters in my reducers, it could be that when it is called the
second time, it overwrites the first results. Oh, wel
The default value for "mapred.reduce.tasks" is indeed "1".
For your cluster, you should tune your client configuration set to
carry a suitable number for that property, in mapred-site.xml
(http://wiki.apache.org/hadoop/HowManyMapsAndReduces might help you
decide how many), or pass it along as a "-
Hi,
I used to do
job.setNumReduceTasks(1);
but I realized that this is bad and commented out this line
//job.setNumReduceTasks(1);
I still see the number of reduce tasks as 1 when my mappers number 4. Why
could this be?
Thank you,
Mark
