Re: jobtracker: Cannot assign requested address
I am still suffering from the problem... Did anyone encounter it before? Or any suggestions? Many thanks in advance! Jing On Fri, Sep 17, 2010 at 5:19 PM, Jing Tie tiej...@gmail.com wrote: Dear all, I am having this exception when starting jobtracker, and I checked by netstat that the port is not in use before running. Could you please point out where might be the problem? Many thanks in advance! 2010-09-17 17:07:22,863 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to hostname/ipaddress:45365 : Cannot assign requested address at org.apache.hadoop.ipc.Server.bind(Server.java:190) at org.apache.hadoop.ipc.Server$Listener.init(Server.java:253) at org.apache.hadoop.ipc.Server.init(Server.java:1026) at org.apache.hadoop.ipc.RPC$Server.init(RPC.java:488) at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450) at org.apache.hadoop.mapred.JobTracker.init(JobTracker.java:1595) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183) at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175) at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702) Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at org.apache.hadoop.ipc.Server.bind(Server.java:188) ... 8 more Configure in mapred-site.xml: property namemapred.job.tracker/name valuehostname:45365/value /property Best, Jing
Re: can not report progress from reducer context with hadoop 0.21
This is a bug in 0.21. MAPREDUCE-1905 (
https://issues.apache.org/jira/browse/MAPREDUCE-1905) is open for this.
On 9/21/10 4:29 PM, Marc Sturlese marc.sturl...@gmail.com wrote:
I am using hadoop 0.21
I have a reducer task wich takes more time to finish that the
mapreduce.task.timeout so it's being killed:
Task attempt_201009211103_0001_r_00_0 failed to report status for 602
seconds. Killing!
I have implemented a thread which is suposed to send progress and update the
status with an incremented counter but it seems not to be working. The
attempt is killed anyway.
I have tried an even simpler example: Not to use a thread and create an
infinite loop in the reducer which updates the status and sends progress on
each iteration... but the attempt keeps being killed:
@Override
public void reduce(Text keyName, IterableText paths, Context context)
throws IOException, InterruptedException {
while(true) {
context.getCounter(COUNTER_ADS.total_ads).increment(1L) ;
context.setStatus(+context.getCounter(COUNTER_ADS.total_ads)) ;
context.progress();
}
context.write(new Text(done!), NullWritable.get());
}
I have even tryed to use TaskInputOutputContext insted of stright Context:
@Override
public void reduce(Text keyName, IterableText paths, Context context)
throws IOException, InterruptedException {
TaskInputOutputContext tac = (TaskInputOutputContext)context;
while(true) {
tac.getCounter(COUNTER_ADS.total_ads).increment(1L) ;
tac.setStatus(+context.getCounter(COUNTER_ADS.total_ads)) ;
tac.progress();
}
context.write(new Text(done!), NullWritable.get());
}
Can anyone tell me what else could I try or what am I doing wrong? I am
really stuck on this problem and have no idea what else to do...
Thanks in advance
--
View this message in context:
http://lucene.472066.n3.nabble.com/can-not-report-progress-from-reducer-context-with-hadoop-0-21-tp1534700p1534700.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: jobtracker: Cannot assign requested address
On 09/21/2010 03:17 AM, Jing Tie wrote: I am still suffering from the problem... Did anyone encounter it before? Or any suggestions? Many thanks in advance! Jing On Fri, Sep 17, 2010 at 5:19 PM, Jing Tietiej...@gmail.com wrote: Dear all, I am having this exception when starting jobtracker, and I checked by netstat that the port is not in use before running. Could you please point out where might be the problem? Many thanks in advance! 2010-09-17 17:07:22,863 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to hostname/ipaddress:45365 : Cannot assign requested address Configure in mapred-site.xml: property namemapred.job.tracker/name valuehostname:45365/value /property Best, Jing Some suggestions: 1) try: property namemapred.job.tracker/name valuehdfs://hostname:45365/value /property 2) Is the port correct? 3) Is the hostname correct? 4) Have you verfified that the host is definitely listening on that port? (e.g., netstat -nap | grep LISTEN) HTH, DR
Re: jobtracker: Cannot assign requested address
I am not sure but try 2 things : 1. Just give your proper Ipaddress in value e, g 123.154.0.122:9001 2. Change the port to 9001 and your etc/hosts file must have these values of master and slaves 123.154.0.122 hostname 123.154.0.111 hostname 123.154.0.112 hostname David Rosenstrauch wrote: On 09/21/2010 03:17 AM, Jing Tie wrote: I am still suffering from the problem... Did anyone encounter it before? Or any suggestions? Many thanks in advance! Jing On Fri, Sep 17, 2010 at 5:19 PM, Jing Tietiej...@gmail.com wrote: Dear all, I am having this exception when starting jobtracker, and I checked by netstat that the port is not in use before running. Could you please point out where might be the problem? Many thanks in advance! 2010-09-17 17:07:22,863 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to hostname/ipaddress:45365 : Cannot assign requested address Configure in mapred-site.xml: property namemapred.job.tracker/name valuehostname:45365/value /property Best, Jing Some suggestions: 1) try: property namemapred.job.tracker/name valuehdfs://hostname:45365/value /property 2) Is the port correct? 3) Is the hostname correct? 4) Have you verfified that the host is definitely listening on that port? (e.g., netstat -nap | grep LISTEN) HTH, DR
Re: can not report progress from reducer context with hadoop 0.21
Thanks, was going mad with this. It's working properly with 0.20.2 Once the patch is totally done will apply it to be able to keep using the MarkableIterator as it simplifies me many MapReduce jobs -- View this message in context: http://lucene.472066.n3.nabble.com/can-not-report-progress-from-reducer-context-with-hadoop-0-21-tp1534700p1555486.html Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
IOException in TaskRunner (Error Code :134)
Hi, Just wanted to know if anyone has any idea about this one? This happens every time I run a job. Is this issue hardware related? Thanks in advance, Krishnakumar. Begin forwarded message: From: C.V.Krishnakumar cvkrishnaku...@me.com Date: September 17, 2010 1:32:49 PM PDT To: common-user@hadoop.apache.org Subject: Tasks Failing : IOException in TaskRunner (Error Code :134) Reply-To: common-user@hadoop.apache.org Hi all, I am facing a problem with the TaskRunner. I have a small hadoop cluster in the fully distributed mode. However when I submit a job, the job never seems to proceed beyond the map 0% reduce 0% stage. Soon after I get this error: java.io.IOException: Task process exit with nonzero status of 134. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418) When I googled this issue, I found this link http://markmail.org/message/lvefqbcboaqzfazt but did not completely understand the issue. The cluster, in this same configuration, was working fine a few days back. So, I am confused as to what could have changed to cause this error. Have any of you faced similar problems too? I would be really grateful if you could let me know if I am missing something very obvious. Thanks a lot! Regards, Krishna.
Re: IOException in TaskRunner (Error Code :134)
It is a not enough information error. Check the tasks, jobtracker, tasktracker, datanode, and namenode logs. On Sep 21, 2010, at 12:30 PM, C.V.Krishnakumar wrote: Hi, Just wanted to know if anyone has any idea about this one? This happens every time I run a job. Is this issue hardware related? Thanks in advance, Krishnakumar. Begin forwarded message: From: C.V.Krishnakumar cvkrishnaku...@me.com Date: September 17, 2010 1:32:49 PM PDT To: common-user@hadoop.apache.org Subject: Tasks Failing : IOException in TaskRunner (Error Code :134) Reply-To: common-user@hadoop.apache.org Hi all, I am facing a problem with the TaskRunner. I have a small hadoop cluster in the fully distributed mode. However when I submit a job, the job never seems to proceed beyond the map 0% reduce 0% stage. Soon after I get this error: java.io.IOException: Task process exit with nonzero status of 134. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418) When I googled this issue, I found this link http://markmail.org/message/lvefqbcboaqzfazt but did not completely understand the issue. The cluster, in this same configuration, was working fine a few days back. So, I am confused as to what could have changed to cause this error. Have any of you faced similar problems too? I would be really grateful if you could let me know if I am missing something very obvious. Thanks a lot! Regards, Krishna.
