[ https://issues.apache.org/jira/browse/MAPREDUCE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ting Dai updated MAPREDUCE-6990: -------------------------------- Description: When file is corrupted, for example, bad encoding, [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the FileInputStream can return 0, causing the while loop in TaskLog$Reader become infinite. {code:java} public Reader(TaskAttemptID taskid, LogName kind, long start, long end, boolean isCleanup) throws IOException { // find the right log file LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup); // calculate the start and stop long size = fileDetail.length; if (start < 0) { start += size + 1; } if (end < 0) { end += size + 1; } start = Math.max(0, Math.min(start, size)); end = Math.max(0, Math.min(end, size)); start += fileDetail.start; end += fileDetail.start; bytesRemaining = end - start; String owner = obtainLogDirOwner(taskid); file = SecureIOUtils.openForRead(new File(fileDetail.location, kind.toString()), owner, null); // skip upto start long pos = 0; while (pos < start) { long result = file.skip(start - pos); if (result < 0) { bytesRemaining = 0; break; } pos += result; } } {code} Similar bugs are [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905) was: When file is corrupted, for example, bad encoding, [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the FileInputStream can return 0, causing the while loop in TaskLog$Reader become infinite. {code:java} // Some comments here public Reader(TaskAttemptID taskid, LogName kind, long start, long end, boolean isCleanup) throws IOException { // find the right log file LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup); // calculate the start and stop long size = fileDetail.length; if (start < 0) { start += size + 1; } if (end < 0) { end += size + 1; } start = Math.max(0, Math.min(start, size)); end = Math.max(0, Math.min(end, size)); start += fileDetail.start; end += fileDetail.start; bytesRemaining = end - start; String owner = obtainLogDirOwner(taskid); file = SecureIOUtils.openForRead(new File(fileDetail.location, kind.toString()), owner, null); // skip upto start long pos = 0; while (pos < start) { long result = file.skip(start - pos); if (result < 0) { bytesRemaining = 0; break; } pos += result; } } {code} Similar bugs are [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905) > FileInputStream.skip function can return 0 when the file is corrupted, > causing an infinite loop > ----------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-6990 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6990 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 0.23.0 > Reporter: Ting Dai > > When file is corrupted, for example, bad encoding, > [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the > FileInputStream can return 0, causing the while loop in TaskLog$Reader become > infinite. > {code:java} > public Reader(TaskAttemptID taskid, LogName kind, long start, long end, > boolean isCleanup) throws IOException { > // find the right log file > LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup); > // calculate the start and stop > long size = fileDetail.length; > if (start < 0) { > start += size + 1; > } > if (end < 0) { > end += size + 1; > } > start = Math.max(0, Math.min(start, size)); > end = Math.max(0, Math.min(end, size)); > start += fileDetail.start; > end += fileDetail.start; > bytesRemaining = end - start; > String owner = obtainLogDirOwner(taskid); > file = SecureIOUtils.openForRead(new File(fileDetail.location, > kind.toString()), owner, null); > // skip upto start > long pos = 0; > while (pos < start) { > long result = file.skip(start - pos); > if (result < 0) { > bytesRemaining = 0; > break; > } > pos += result; > } > } > {code} > Similar bugs are > [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), > [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905) -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org