Re: Map Task Capacity Not Changing
To follow up on what I have found: I opened up some of the logs on the datanodes and found this message: Can not start task tracker because java.net.BindException: Address already in use It was using the default port setting from mapred-default.xml, which was 50060. I decided to try an add property namemapred.task.tracker.http.address/name value0.0.0.0:0/value /property to mapred-site.xml so that the first open port would be selected. This works and it also, allows for the tasktracker to start normally which in turn allows the mapred.tasktracker.map.tasks.maximum setting to take effect. Thanks for all who helped. --Joey On Sat, Dec 17, 2011 at 1:42 AM, Joey Krabacher jkrabac...@gmail.com wrote: pid files are there, I checked for running processes with the sameID's and they all checked out. --Joey On Fri, Dec 16, 2011 at 5:40 PM, Rahul Jain rja...@gmail.com wrote: You might be suffering from HADOOP-7822; I'd suggest you verify your pid files and fix the problem by hand if it is the same issue. -Rahul On Fri, Dec 16, 2011 at 2:40 PM, Joey Krabacher jkrabac...@gmail.comwrote: Turns out my tasktrackers(on the datanodes) are not starting properly so I guess they are taking some alternate route?? because they are up and running...even though when I run stop-mapred.sh it says data01: no tasktracker to stop --Joey On Thu, Dec 15, 2011 at 5:37 PM, James Warren james.war...@stanfordalumni.org wrote: (moving to mapreduce-user@, bcc'ing common-user@) Hi Joey - You'll want to change the value on all of your servers running tasktrackers and then restart each tasktracker to reread the configuration. cheers, -James On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher jkrabac...@gmail.com wrote: I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Re: Map Task Capacity Not Changing
You might be suffering from HADOOP-7822; I'd suggest you verify your pid files and fix the problem by hand if it is the same issue. -Rahul On Fri, Dec 16, 2011 at 2:40 PM, Joey Krabacher jkrabac...@gmail.comwrote: Turns out my tasktrackers(on the datanodes) are not starting properly so I guess they are taking some alternate route?? because they are up and running...even though when I run stop-mapred.sh it says data01: no tasktracker to stop --Joey On Thu, Dec 15, 2011 at 5:37 PM, James Warren james.war...@stanfordalumni.org wrote: (moving to mapreduce-user@, bcc'ing common-user@) Hi Joey - You'll want to change the value on all of your servers running tasktrackers and then restart each tasktracker to reread the configuration. cheers, -James On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher jkrabac...@gmail.com wrote: I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Re: Map Task Capacity Not Changing
pid files are there, I checked for running processes with the same ID's and they all checked out. --Joey On Fri, Dec 16, 2011 at 5:40 PM, Rahul Jain rja...@gmail.com wrote: You might be suffering from HADOOP-7822; I'd suggest you verify your pid files and fix the problem by hand if it is the same issue. -Rahul On Fri, Dec 16, 2011 at 2:40 PM, Joey Krabacher jkrabac...@gmail.comwrote: Turns out my tasktrackers(on the datanodes) are not starting properly so I guess they are taking some alternate route?? because they are up and running...even though when I run stop-mapred.sh it says data01: no tasktracker to stop --Joey On Thu, Dec 15, 2011 at 5:37 PM, James Warren james.war...@stanfordalumni.org wrote: (moving to mapreduce-user@, bcc'ing common-user@) Hi Joey - You'll want to change the value on all of your servers running tasktrackers and then restart each tasktracker to reread the configuration. cheers, -James On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher jkrabac...@gmail.com wrote: I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Re: Map Task Capacity Not Changing
pid files are there, I checked for running processes with the sameID's and they all checked out. --Joey On Fri, Dec 16, 2011 at 5:40 PM, Rahul Jain rja...@gmail.com wrote: You might be suffering from HADOOP-7822; I'd suggest you verify your pid files and fix the problem by hand if it is the same issue. -Rahul On Fri, Dec 16, 2011 at 2:40 PM, Joey Krabacher jkrabac...@gmail.comwrote: Turns out my tasktrackers(on the datanodes) are not starting properly so I guess they are taking some alternate route?? because they are up and running...even though when I run stop-mapred.sh it says data01: no tasktracker to stop --Joey On Thu, Dec 15, 2011 at 5:37 PM, James Warren james.war...@stanfordalumni.org wrote: (moving to mapreduce-user@, bcc'ing common-user@) Hi Joey - You'll want to change the value on all of your servers running tasktrackers and then restart each tasktracker to reread the configuration. cheers, -James On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher jkrabac...@gmail.com wrote: I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Map Task Capacity Not Changing
I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Re: Map Task Capacity Not Changing
(moving to mapreduce-user@, bcc'ing common-user@) Hi Joey - You'll want to change the value on all of your servers running tasktrackers and then restart each tasktracker to reread the configuration. cheers, -James On Thu, Dec 15, 2011 at 3:30 PM, Joey Krabacher jkrabac...@gmail.comwrote: I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
Re: Map Task Capacity Not Changing
Joey What is it do you really want to do? Increase the number of map slots available in the task tracker or do you want to increase the number of map tasks for a job? If you want to increase the number of map slots available, what you did will work - as long as you restarted the task tracker or the node. If you want to increase the number of map jobs - you will have to play with, simplistically speaking, either the number of input files or input block size. Raj From: Joey Krabacher jkrabac...@gmail.com To: common-user@hadoop.apache.org Sent: Thursday, December 15, 2011 3:30 PM Subject: Map Task Capacity Not Changing I have looked up how to up this value on the web and have tried all suggestions to no avail. Any help would be great. Here is some background: Version: 0.20.2, r911707 Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo Nodes: 5 Current Map Task Capacity : 10 --- this is what I want to increase. What I have tried : Adding property namemapred.tasktracker.map.tasks.maximum/name value8/value finaltrue/final /property to mapred-site.xml on NameNode. I also added this to one of the datanodes for the hell of it and that didn't work either. Thanks.
