Re: Problems increasing number of tasks per node- really a task management problem!

2008-09-23 Thread Joel Welling
I think I've found my problem. At some point about a week ago, I must have tried to start new tasktracker processes on my worker nodes without killing the ones that were already there. The new processes died immediately because their sockets were already in use. The old processes then took over

Re: Problems increasing number of tasks per node

2008-09-23 Thread Arun C Murthy
On Sep 23, 2008, at 2:21 PM, Joel Welling wrote: Stopping and restarting the mapred service should push the new .xml file out, should it not? I've done 'bin/mapred-stop.sh', No, you need to run 'bin/mapred-stop.sh', push it out to all the machines and then do 'bin/mapred-start.sh'. You

Re: Problems increasing number of tasks per node

2008-09-23 Thread Joel Welling
Stopping and restarting the mapred service should push the new .xml file out, should it not? I've done 'bin/mapred-stop.sh', 'bin/mapred-start.sh', and I can see my new values in the file:.../mapred/system/job_SomeNumber_SomeNumber/job.xml files associated with the jobs. The mapred.tasktracker.ma

Re: Problems increasing number of tasks per node

2008-09-23 Thread Arun C Murthy
On Sep 23, 2008, at 11:41 AM, Joel Welling wrote: Hi folks; I have a small cluster, but each node is big- 8 cores each, with lots of IO bandwidth. I'd like to increase the number of simultaneous map and reduce tasks scheduled per node from the default of 2 to something like 8. My understandi

Problems increasing number of tasks per node

2008-09-23 Thread Joel Welling
Hi folks; I have a small cluster, but each node is big- 8 cores each, with lots of IO bandwidth. I'd like to increase the number of simultaneous map and reduce tasks scheduled per node from the default of 2 to something like 8. My understanding is that I should be able to do this by increasing