Hi ,All,
I encountered a problem in using Cloudera Hadoop 0.20.2-cdh3u1. When I use
the fair Scheduler I find the scheduler seems not support preemption.
Can anybody tell me whether preemption is supported in this version?
This is my configration:
mapred-site.xml
<property>
<name>mapred.jobtracker.taskScheduler</name>
<value>org.apache.hadoop.mapred.FairScheduler</value>
</property>
<property>
<name>mapred.fairscheduler.allocation.file</name>
<value>/usr/lib/hadoop-0.20/conf/fair-scheduler.xml</value>
</property>
<property>
<name>mapred.fairscheduler.preemption</name>
<value>true</value>
</property>
<property>
<name>mapred.fairscheduler.preemption.only.log</name>
<value>true</value>
</property>
<property>
<name>mapred.fairscheduler.preemption.interval</name>
<value>15000</value>
</property>
<property>
<name>mapred.fairscheduler.weightadjuster</name>
<value>org.apache.hadoop.mapred.NewJobWeightBooster</value>
</property>
<property>
<name>mapred.fairscheduler.sizebasedweight</name>
<value>true</value>
</property>
fair-scheduler.xml
<allocations>
<pool name="root">
<minMaps>10</minMaps>
<minReduces>5</minReduces>
<maxMaps>200</maxMaps>
<maxReduces>80</maxReduces>
<maxRunningJobs>100</maxRunningJobs>
<minSharePreemptionTimeout>30</minSharePreemptionTimeout>
<weight>1.0</weight>
</pool>
<pool name="hadoop">
<minMaps>10</minMaps>
<minReduces>5</minReduces>
<maxMaps>80</maxMaps>
<maxReduces>80</maxReduces>
<maxRunningJobs>5</maxRunningJobs>
<minSharePreemptionTimeout>30</minSharePreemptionTimeout>
<weight>1.0</weight>
</pool>
<user name="user1">
<maxRunningJobs>10</maxRunningJobs>
</user>
<poolMaxJobsDefault>20</poolMaxJobsDefault>
<userMaxJobsDefault>10</userMaxJobsDefault>
<defaultMinSharePreemptionTimeout>30</defaultMinSharePreemptionTimeout>
<fairSharePreemptionTimeout>30</fairSharePreemptionTimeout>
</allocations>
regards,
2012-03-07
hao.wang
