The properties mapred.tasktracker.{map,reduce}.tasks.maximum are node level
parameters, and are read by tasktrackers when they start up. Passing them in
the job confing doesn't do any change ...
On Tue, Jul 5, 2011 at 12:02 PM, Saket Joshi wrote:
> Praveen,
>
> ** **
>
> You also need to se
Saket,
Do note that while FileInputFormats and most of its derivatives would respect
the "mapred.map.tasks" as a property to get a desired number of splits [1],
other formats may not consider it at all (i.e. you need to check sources to
make sure they can).
[1] - The getSplits(…) interface of
Praveen,
You also need to set the property "mapred.map.tasks" to the value derived
from the following formula ( mapred.tasktracker.map.tasks.maximum * Number of
tasktracker Nodes in your cluster) . I am using Hadoop 0.20.2 and I am able to
increase the number of maps using these two mentoined
If your datanode has 2 HDFS-chunks (blocks) of the input file, the scheduler
will first prefer to run 2 map tasks on the tasktracker where this datanode
resides.
On Fri, Jul 1, 2011 at 10:33 PM, Juwei Shi wrote:
> I think that Anthony is right. Task capacity has to been set at
> mapred-default.
I think that Anthony is right. Task capacity has to been set at
mapred-default.html, and restart the cluster.
Anthony Urso
2011/7/2
> Are you sure? AFAIK all mapred.xxx properties can be set via job config. I
> also read on yahoo tutorial that this property can be either set in
> hadoop-site
This property applies to a tasktraker rather that an individual job.
Therefore it needs to be set in the mapred-site.xml and the daemon
restarted.
-Joey
On Jul 1, 2011 7:01 PM, wrote:
> Are you sure? AFAIK all mapred.xxx properties can be set via job config. I
also read on yahoo tutorial that thi
Are you sure? AFAIK all mapred.xxx properties can be set via job config. I also
read on yahoo tutorial that this property can be either set in hadoop-site.XML
or job config. May be someone can confirm this who have really used this
property.
Praveen
On Jul 1, 2011, at 4:46 PM, "ext Anthony Urs
On Fri, Jul 1, 2011 at 1:03 PM, wrote:
> Hi all,
>
> I am using hadoop 0.20.2. I am setting the property
> mapred.tasktracker.map.tasks.maximum = 4 (same for reduce also) on my job
> conf but I am still seeing max of only 2 map and reduce tasks on each node.
> I know my machine can run 4 maps and
