RE: Taktracker in namenode failure
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class. But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring. The problem is only with the tasktracker running in the namenode. Thanks Regards Ramya.S From: Devaraj k [mailto:devara...@huawei.com] Sent: Fri 7/12/2013 3:04 PM To: user@hadoop.apache.org Subject: RE: Taktracker in namenode failure Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error. Thanks Devaraj k From: Ramya S [mailto:ram...@suntecgroup.com] Sent: 12 July 2013 14:46 To: user@hadoop.apache.org Subject: Taktracker in namenode failure Hi, Why only tasktracker in namenode faill during job execution with error. I have attached the snapshot of error screen with this mail java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at WordCount$TokenizerMapper.map(WordCount.java:30) at WordCount$TokenizerMapper.map(WordCount.java:19) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) but this same task is reassigned to another tasktracker and getting executed. why? Best Regards, Ramya winmail.dat
RE: Taktracker in namenode failure
I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job. Thanks Devaraj k From: Ramya S [mailto:ram...@suntecgroup.com] Sent: 12 July 2013 15:27 To: user@hadoop.apache.org Subject: RE: Taktracker in namenode failure Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class. But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring. The problem is only with the tasktracker running in the namenode. Thanks Regards Ramya.S From: Devaraj k [mailto:devara...@huawei.com] Sent: Fri 7/12/2013 3:04 PM To: user@hadoop.apache.orgmailto:user@hadoop.apache.org Subject: RE: Taktracker in namenode failure Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error. Thanks Devaraj k From: Ramya S [mailto:ram...@suntecgroup.com] Sent: 12 July 2013 14:46 To: user@hadoop.apache.orgmailto:user@hadoop.apache.org Subject: Taktracker in namenode failure Hi, Why only tasktracker in namenode faill during job execution with error. I have attached the snapshot of error screen with this mail java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at WordCount$TokenizerMapper.map(WordCount.java:30) at WordCount$TokenizerMapper.map(WordCount.java:19) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) but this same task is reassigned to another tasktracker and getting executed. why? Best Regards, Ramya
Re: Taktracker in namenode failure
Hi, The problem is with jar file only, to check run any other MR job or sample wordcount job on namenode tasktracker, if it is running no problem with namenode tasktracker, if not running there may be problem with tasktracker configuration, then compare with other node tasktracker configuration. i.e tasktracker configuration means mapred configuration. Hi, From, Ramesh. On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k devara...@huawei.com wrote: I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job. ** ** Thanks Devaraj k ** ** *From:* Ramya S [mailto:ram...@suntecgroup.com] *Sent:* 12 July 2013 15:27 *To:* user@hadoop.apache.org *Subject:* RE: Taktracker in namenode failure ** ** Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class. But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring. The problem is only with the tasktracker running in the namenode. *Thanks Regards* *Ramya.S* ** ** -- *From:* Devaraj k [mailto:devara...@huawei.com devara...@huawei.com] *Sent:* Fri 7/12/2013 3:04 PM *To:* user@hadoop.apache.org *Subject:* RE: Taktracker in namenode failure Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error. Thanks Devaraj k *From:* Ramya S [mailto:ram...@suntecgroup.com ram...@suntecgroup.com] *Sent:* 12 July 2013 14:46 *To:* user@hadoop.apache.org *Subject:* Taktracker in namenode failure Hi, Why only tasktracker in namenode faill during job execution with error.** ** I have attached the snapshot of error screen with this mail java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at WordCount$TokenizerMapper.map(WordCount.java:30) at WordCount$TokenizerMapper.map(WordCount.java:19) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) but this same task is reassigned to another tasktracker and getting executed. why? *Best Regards,* *Ramya*
