[ http://issues.apache.org/jira/browse/HADOOP-18?page=all ]
Doug Cutting updated HADOOP-18:
-------------------------------
Component: mapred
> Crash with multiple temp directories
> ------------------------------------
>
> Key: HADOOP-18
> URL: http://issues.apache.org/jira/browse/HADOOP-18
> Project: Hadoop
> Type: Bug
> Components: mapred
> Reporter: Rod Taylor
> Priority: Critical
>
> A brief read of the code indicated it may be possible to use multiple local
> directories using something like the below:
> <property>
> <name>mapred.local.dir</name>
> <value>/local,/local1,/local2</value>
> <description>The local directory where MapReduce stores intermediate
> data files.
> </description>
> </property>
> This failed with the below exception during either the generate or update
> phase (not entirely sure which).
> java.lang.ArrayIndexOutOfBoundsException
> at java.util.zip.CRC32.update(CRC32.java:51)
> at
> org.apache.nutch.fs.NFSDataInputStream$Checker.read(NFSDataInputStream.java:92)
> at
> org.apache.nutch.fs.NFSDataInputStream$PositionCache.read(NFSDataInputStream.java:156)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
> at java.io.DataInputStream.readFully(DataInputStream.java:176)
> at
> org.apache.nutch.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:55)
> at
> org.apache.nutch.io.DataOutputBuffer.write(DataOutputBuffer.java:89)
> at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:378)
> at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:301)
> at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:323)
> at
> org.apache.nutch.mapred.SequenceFileRecordReader.next(SequenceFileRecordReader.java:60)
> at
> org.apache.nutch.segment.SegmentReader$InputFormat$1.next(SegmentReader.java:80)
> at org.apache.nutch.mapred.MapTask$2.next(MapTask.java:106)
> at org.apache.nutch.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.nutch.mapred.MapTask.run(MapTask.java:116)
> at
> org.apache.nutch.mapred.TaskTracker$Child.main(TaskTracker.java:604)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
