Hadoop-0.14 introduced job priorities (https://issues.apache.org/jira/
browse/HADOOP-1433); you might be able to get somewhere with this.
Another possibility is to create two mapreduce clusters on top of the
same dfs cluster.
The mapred.tasktracker.tasks.maximum doesn't do what you think -- it
actually controls the numbers of tasks that run simultaneously on a
tasktracker machine.
-Michael
On Jan 9, 2008, at 11:25 AM, Xavier Stevens wrote:
Does Hadoop support running simultaneous jobs? If so, what parameters
do I need to set in my job configuration? We basically want to give a
job that takes a really long time, half of the total resources of the
cluster so other jobs don't queue up behind it.
I am using Hadoop 0.14.2 currently. I tried setting
mapred.tasktracker.tasks.maximum to be half of the maximum
specified in
mapred-default.xml. This shows the change in the web administration
page for the job, but it has no effect on the actual numbers of tasks
running.
Thanks,
Xavier
