My cluster nodes have 2 dual-core processors, so, in general, I want to configure my nodes with a maximum of 3 task processes executed per node at a time.
But, for some jobs, my tasks load large amounts of memory, and I cannot fit 3 such tasks on a single node. For these jobs, I'd like to enforce running a maximum of 1 task process per node at a time. I've tried to enforce this by setting mapred.tasktracker.map.tasks.maximum at runtime, but I see it has no effect, because this is a configuration for the TaskTracker, which is of course already running before my job starts. Is there no way to configure a maximum # of map tasks per node on a per-job basis? Thanks!
