[
https://issues.apache.org/jira/browse/MAPREDUCE-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840404#action_12840404
]
Allen Wittenauer commented on MAPREDUCE-1552:
---------------------------------------------
Here is the stack trace we get in the attempt log:
2010-03-02 13:04:11,182 INFO org.apache.hadoop.mapred.Merger: Merging 115
sorted segments
2010-03-02 13:04:11,852 INFO org.apache.hadoop.mapred.Merger: Merging 16
intermediate segments out of a total of 115
2010-03-02 13:05:13,784 FATAL org.apache.hadoop.mapred.TaskTracker: FSError
from child
org.apache.hadoop.fs.FSError: java.io.IOException: Disc quota exceeded
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:192)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
...
Caused by: java.io.IOException: Disc quota exceeded
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:190)
... 17 more
The location might be in the task tracker log, but it seems like it would be
more obvious to be here.
> TaskTracker should report which fs during error
> -----------------------------------------------
>
> Key: MAPREDUCE-1552
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1552
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
>
> We run with ZFS with fs quotas for the mapred spill space to prevent it
> over-running the HDFS space. During merge, we some times end up running out
> of space. It would be useful if the stack trace (see below) included which
> file system the errors actually came from.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
