Hi, Thanks for your reply! I have solved this problem by setting "mapred.fairscheduler.preemption.only.log " to "false". The preemption works! But I don't know why can not set "mapred.fairscheduler.preemption.only.log " to "true". Is it a bug?
regards, 2012-03-07 hao.wang 发件人： Harsh J 发送时间： 2012-03-07 14:14:05 收件人： common-user 抄送： 主题： Re: Fair Scheduler Problem Hello Hao, Its best to submit CDH user queries to https://groups.google.com/a/cloudera.org/group/cdh-user/topics (cdh-u...@cloudera.org) where the majority of CDH users community resides. How do you determine that preemption did not/does not work? Preemption between pools occurs if a pool's minShare isn't satisfied within preemption-timeout seconds. In this case, it will preempt tasks from other pools. Your settings look alright on a high level. Does your log not carry any preemption logs? What was your pool's share scenario when you tried to observe if it works or not? On Wed, Mar 7, 2012 at 8:35 AM, hao.wang <hao.w...@ipinyou.com> wrote: > Hi ,All, > I encountered a problem in using Cloudera Hadoop 0.20.2-cdh3u1. When I use > the fair Scheduler I find the scheduler seems not support preemption. > Can anybody tell me whether preemption is supported in this version? > This is my configration: > mapred-site.xml > <property> > <name>mapred.jobtracker.taskScheduler</name> > <value>org.apache.hadoop.mapred.FairScheduler</value> > </property> > <property> > <name>mapred.fairscheduler.allocation.file</name> > <value>/usr/lib/hadoop-0.20/conf/fair-scheduler.xml</value> > </property> > <property> > <name>mapred.fairscheduler.preemption</name> > <value>true</value> > </property> > <property> > <name>mapred.fairscheduler.preemption.only.log</name> > <value>true</value> > </property> > <property> > <name>mapred.fairscheduler.preemption.interval</name> > <value>15000</value> > </property> > <property> > <name>mapred.fairscheduler.weightadjuster</name> > <value>org.apache.hadoop.mapred.NewJobWeightBooster</value> > </property> > <property> > <name>mapred.fairscheduler.sizebasedweight</name> > <value>true</value> > </property> > fair-scheduler.xml > <allocations> > <pool name="root"> > <minMaps>10</minMaps> > <minReduces>5</minReduces> > <maxMaps>200</maxMaps> > <maxReduces>80</maxReduces> > <maxRunningJobs>100</maxRunningJobs> > <minSharePreemptionTimeout>30</minSharePreemptionTimeout> > <weight>1.0</weight> > </pool> > <pool name="hadoop"> > <minMaps>10</minMaps> > <minReduces>5</minReduces> > <maxMaps>80</maxMaps> > <maxReduces>80</maxReduces> > <maxRunningJobs>5</maxRunningJobs> > <minSharePreemptionTimeout>30</minSharePreemptionTimeout> > <weight>1.0</weight> > </pool> > <user name="user1"> > <maxRunningJobs>10</maxRunningJobs> > </user> > <poolMaxJobsDefault>20</poolMaxJobsDefault> > <userMaxJobsDefault>10</userMaxJobsDefault> > <defaultMinSharePreemptionTimeout>30</defaultMinSharePreemptionTimeout> > <fairSharePreemptionTimeout>30</fairSharePreemptionTimeout> > </allocations> > > regards, > > 2012-03-07 > > > > hao.wang -- Harsh J
