It hasn't been cut as a release, but there's a 19.2 development branch at branches/branch-19 - Aaron
On Fri, May 22, 2009 at 2:31 PM, Lance Riedel <la...@dotspots.com> wrote: > Sure, I'll try out 19.2.. but where is it? I don't see it here: > http://svn.apache.org/repos/asf/hadoop/core/ > (looking under tags) > > > > On Fri, May 22, 2009 at 2:11 PM, Todd Lipcon <t...@cloudera.com> wrote: > > > Hi Lance, > > > > It's possible this is related to the other JIRA (HADOOP-5761). If it's > not > > too much trouble to try out the 19.2 branch from SVN, it would be helpful > > in > > determining whether this is a problem that's already fixed or if you've > > discovered something new. > > > > Thanks > > -Todd > > > > On Fri, May 22, 2009 at 2:01 PM, Lance Riedel <la...@dotspots.com> > wrote: > > > > > Hi Todd, > > > We had looked at that before.. here is the location of the tmp > directory: > > > > > > [dotsp...@domu-12-31-38-00-80-21 hadoop-0.19.1]$ du -sh > > > /dist/app/hadoop-0.19.1/tmp > > > 248G /dist/app/hadoop-0.19.1/tmp > > > > > > There are no cron jobs that would have anything to do with that > > directory. > > > > > > Here is the /tmp > > > [dotsp...@domu-12-31-38-00-80-21 hadoop-0.19.1]$ du -sh /tmp > > > 204K /tmp > > > > > > Does this look like a disk error? I had seen that the > > > "org.apache.hadoop.util.DiskChecker$DiskErrorException" is bogus. > > > > > > Thanks! > > > Lance > > > > > > > > > > > > > > > > > > On Fri, May 22, 2009 at 9:33 AM, Lance Riedel <la...@dotspots.com> > > wrote: > > > > > > > Version 19.1 with patches: > > > > 4780-2v19.patch (Jira 4780) > > > > closeAll3.patch (Jira 3998) > > > > I have confirmed that > > > https://issues.apache.org/jira/browse/HADOOP-4924patch is in, so that > is > > > not the fix. > > > > > > > > > > > > We are having task trackers die every night with a null pointer > > > exception. > > > > Usually 2 or so out of 8 (25% each night). > > > > > > > > > > > > Here are the logs: > > > > > > > > Version 19.1 with > > > > 2009-05-22 02:46:49,911 INFO org.apache.hadoop.mapred.TaskTracker: > > > > Received 'KillJobAction' for job: job_200905211749_0451 > > > > 2009-05-22 02:46:49,911 INFO org.apache.hadoop.mapred.TaskRunner: > > > > attempt_200905211749_0451_m_000000_0 done; removing files. > > > > 2009-05-22 02:46:54,911 INFO org.apache.hadoop.mapred.TaskTracker: > > > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > > > > > > > > taskTracker/jobcache/job_200905211749_0421/attempt_200905211749_0421_r_000009_0/output/file.out > > > > in any of the configured local directories > > > > 2009-05-22 02:47:13,968 INFO org.apache.hadoop.mapred.TaskTracker: > > > Received > > > > 'KillJobAction' for job: job_200905211749_0444 > > > > 2009-05-22 02:47:13,969 INFO org.apache.hadoop.mapred.TaskRunner: > > > > attempt_200905211749_0444_m_000000_0 done; removing files. > > > > 2009-05-22 02:47:18,968 INFO org.apache.hadoop.mapred.TaskTracker: > > > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > > > > > > > > taskTracker/jobcache/job_200905211749_0421/attempt_200905211749_0421_r_000009_0/output/file.out > > > > in any of the configured local directories > > > > 2009-05-22 02:48:52,324 INFO org.apache.hadoop.mapred.TaskTracker: > > > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > > > > > > > > taskTracker/jobcache/job_200905211749_0421/attempt_200905211749_0421_r_000009_0/output/file.out > > > > in any of the configured local directories > > > > 2009-05-22 02:49:10,779 INFO org.apache.hadoop.mapred.TaskTracker: > > > > LaunchTaskAction (registerTask): attempt_200905211749_0452_m_000006_0 > > > task's > > > > state:UNASSIGNED > > > > 2009-05-22 02:49:10,779 INFO org.apache.hadoop.mapred.TaskTracker: > > Trying > > > > to launch : attempt_200905211749_0452_m_000006_0 > > > > 2009-05-22 02:49:10,779 INFO org.apache.hadoop.mapred.TaskTracker: In > > > > TaskLauncher, current free slots : 4 and trying to launch > > > > attempt_200905211749_0452_m_000006_0 > > > > 2009-05-22 02:49:15,274 INFO org.apache.hadoop.mapred.JvmManager: JVM > > > > Runner jvm_200905211749_0452_m_1998728288 spawned. > > > > 2009-05-22 02:49:15,765 INFO org.apache.hadoop.mapred.TaskTracker: > JVM > > > with > > > > ID: jvm_200905211749_0452_m_1998728288 given task: > > > > attempt_200905211749_0452_m_000006_0 > > > > 2009-05-22 02:49:15,781 INFO org.apache.hadoop.mapred.TaskTracker: > > > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > > > > > > > > taskTracker/jobcache/job_200905211749_0421/attempt_200905211749_0421_r_000009_0/output/file.out > > > > in any of the configured local directories > > > > 2009-05-22 02:49:15,781 INFO org.apache.hadoop.mapred.TaskTracker: > > > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > > > > > > > > taskTracker/jobcache/job_200905211749_0452/attempt_200905211749_0452_m_000006_0/output/file.out > > > > in any of the configured local directories > > > > 2009-05-22 02:49:19,784 INFO org.apache.hadoop.mapred.TaskTracker: > > > > attempt_200905211749_0452_m_000006_0 1.0% hdfs:// > > > > > > > > > > ec2-75-101-247-52.compute-1.amazonaws.com:54310/paragraphInstances/2009-05-22/rollup#06h#3e04c188-245a-4856-9a54-2fec60e85e3d.seq:0+9674259 > > > < > > > > > > http://ec2-75-101-247-52.compute-1.amazonaws.com:54310/paragraphInstances/2009-05-22/rollup#06h%233e04c188-245a-4856-9a54-2fec60e85e3d.seq:0+9674259 > > > > > > > > 2009-05-22 02:49:19,785 INFO org.apache.hadoop.mapred.TaskTracker: > Task > > > > attempt_200905211749_0452_m_000006_0 is done. > > > > 2009-05-22 02:49:19,785 INFO org.apache.hadoop.mapred.TaskTracker: > > > reported > > > > output size for attempt_200905211749_0452_m_000006_0 was 0 > > > > 2009-05-22 02:49:19,787 INFO org.apache.hadoop.mapred.TaskTracker: > > > > addFreeSlot : current free slots : 4 > > > > 2009-05-22 02:49:19,954 INFO org.apache.hadoop.mapred.JvmManager: JVM > : > > > > jvm_200905211749_0452_m_1998728288 exited. Number of tasks it ran: 1 > > > > 2009-05-22 02:59:19,297 INFO org.apache.hadoop.mapred.TaskTracker: > > > Recieved > > > > RenitTrackerAction from JobTracker > > > > 2009-05-22 02:59:19,298 ERROR org.apache.hadoop.mapred.TaskTracker: > Can > > > not > > > > start task tracker because java.lang.NullPointerException > > > > at > > > > > > > > > > org.apache.hadoop.mapred.TaskTracker$TaskInProgress.kill(TaskTracker.java:2300) > > > > at > > > > > > > > > > org.apache.hadoop.mapred.TaskTracker$TaskInProgress.jobHasFinished(TaskTracker.java:2273) > > > > at > > > org.apache.hadoop.mapred.TaskTracker.close(TaskTracker.java:840) > > > > at > > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1728) > > > > at > > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2785) > > > > > > > > 2009-05-22 02:59:19,300 INFO org.apache.hadoop.mapred.TaskTracker: > > > > SHUTDOWN_MSG: > > > > /************************************************************ > > > > SHUTDOWN_MSG: Shutting down TaskTracker at domU-12-31-38-01-AD-91/ > > > > 10.253.178.95 > > > > ************************************************************/ > > > > > > > > > > > > > >
