DataNode not able to spawn a Task
Hi guys, I see the exception below when I launch a job 0/04/27 10:54:16 INFO mapred.JobClient: map 0% reduce 0% 10/04/27 10:54:22 INFO mapred.JobClient: Task Id : attempt_201004271050_0001_m_005760_0, Status : FAILED Error initializing attempt_201004271050_0001_m_005760_0: java.lang.NumberFormatException: For input string: - at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:476) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.hadoop.fs.DF.parseExecResult(DF.java:125) at org.apache.hadoop.util.Shell.runCommand(Shell.java:179) at org.apache.hadoop.util.Shell.run(Shell.java:134) at org.apache.hadoop.fs.DF.getAvailable(DF.java:73) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:751) at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1665) at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:97) at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1630) Few things * I ran fsck on the namenode and no corrupted blocks reported. * The -report from dfsadmin , says the datanode is up. -- View this message in context: http://old.nabble.com/DataNode-not-able-to-spawn-a-Task-tp28378863p28378863.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
Re: DataNode not able to spawn a Task
Hi Vishal, What operating system are you on? The TT is having issues parsing the output of df -Todd On Tue, Apr 27, 2010 at 9:03 AM, vishalsant vishal.santo...@gmail.comwrote: Hi guys, I see the exception below when I launch a job 0/04/27 10:54:16 INFO mapred.JobClient: map 0% reduce 0% 10/04/27 10:54:22 INFO mapred.JobClient: Task Id : attempt_201004271050_0001_m_005760_0, Status : FAILED Error initializing attempt_201004271050_0001_m_005760_0: java.lang.NumberFormatException: For input string: - at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:476) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.hadoop.fs.DF.parseExecResult(DF.java:125) at org.apache.hadoop.util.Shell.runCommand(Shell.java:179) at org.apache.hadoop.util.Shell.run(Shell.java:134) at org.apache.hadoop.fs.DF.getAvailable(DF.java:73) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:751) at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1665) at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:97) at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1630) Few things * I ran fsck on the namenode and no corrupted blocks reported. * The -report from dfsadmin , says the datanode is up. -- View this message in context: http://old.nabble.com/DataNode-not-able-to-spawn-a-Task-tp28378863p28378863.html Sent from the Hadoop core-user mailing list archive at Nabble.com. -- Todd Lipcon Software Engineer, Cloudera
Re: DataNode not able to spawn a Task
It seems that this piece of code , does a df to get the amount of free space ( got this info from the irc channel ) And it is trying to do a Number conversion on information returned by df /Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda21891213200 -45291780 1838887216 - / . . Of course in my case the Use% is - and that is an issue :) BTW , this datanode had stopped responding.. it is always good idea to do df thus , to make sure that this does not happen during job execution and may be even as a part of the ./hadoop dfsadmin -report pbly. Will close the thread , when this is resolved with the disk issue ( which it seems to be ). vishalsant wrote: Hi guys, I see the exception below when I launch a job 0/04/27 10:54:16 INFO mapred.JobClient: map 0% reduce 0% 10/04/27 10:54:22 INFO mapred.JobClient: Task Id : attempt_201004271050_0001_m_005760_0, Status : FAILED Error initializing attempt_201004271050_0001_m_005760_0: java.lang.NumberFormatException: For input string: - at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:476) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.hadoop.fs.DF.parseExecResult(DF.java:125) at org.apache.hadoop.util.Shell.runCommand(Shell.java:179) at org.apache.hadoop.util.Shell.run(Shell.java:134) at org.apache.hadoop.fs.DF.getAvailable(DF.java:73) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:751) at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1665) at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:97) at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1630) Few things * I ran fsck on the namenode and no corrupted blocks reported. * The -report from dfsadmin , says the datanode is up. -- View this message in context: http://old.nabble.com/DataNode-not-able-to-spawn-a-Task-tp28378863p28379065.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
