[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414358#comment-13414358 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1103 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1103/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 2.0.0-alpha Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414412#comment-13414412 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1136 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1136/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 2.0.0-alpha Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414170#comment-13414170 ] Colin Patrick McCabe commented on HDFS-3492: +1 LGTM fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414234#comment-13414234 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2488 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2488/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 2.0.0-alpha Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414257#comment-13414257 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2469 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2469/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 2.0.0-alpha Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414262#comment-13414262 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2534 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2534/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 2.0.0-alpha Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413428#comment-13413428 ] Hadoop QA commented on HDFS-3492: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536311/hdfs-3492.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.common.TestJspHelper +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2809//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2809//console This message is automatically generated. fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291722#comment-13291722 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1071 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1071/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291772#comment-13291772 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1104 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1104/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290937#comment-13290937 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1070 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1070/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 2.0.1-alpha Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291005#comment-13291005 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1103 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1103/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 2.0.1-alpha Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291311#comment-13291311 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2332 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2332/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291314#comment-13291314 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2405 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2405/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291371#comment-13291371 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2351 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2351/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291384#comment-13291384 ] Colin Patrick McCabe commented on HDFS-3492: It looks like it's the change to BlockReadeLocal.java that caused the test failure here. I'm still not sure about why. fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291543#comment-13291543 ] Hadoop QA commented on HDFS-3492: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531333/HDFS-3492.002.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.datanode.TestBPOfferService org.apache.hadoop.test.TestHFSTestCase org.apache.hadoop.test.TestHTestCase +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2617//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2617//console This message is automatically generated. fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290527#comment-13290527 ] Todd Lipcon commented on HDFS-3492: --- +1, looks good to me. Thanks Colin. fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290553#comment-13290553 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2329 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2329/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 2.0.1-alpha Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290560#comment-13290560 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2402 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2402/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 2.0.1-alpha Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290615#comment-13290615 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2348 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2348/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 2.0.1-alpha Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13288253#comment-13288253 ] Hadoop QA commented on HDFS-3492: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530706/HDFS-3492.001.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2578//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2578//console This message is automatically generated. fix some misuses of InputStream#skip Key: HDFS-3492 URL: https://issues.apache.org/jira/browse/HDFS-3492 Project: Hadoop HDFS Issue Type: Bug Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Attachments: HDFS-3492.001.patch It seems that we have a few cases where programmers are calling InputStream#skip and not handling short skips. Unfortunately, the skip method is documented and implemented so that it doesn't actually skip the requested number of bytes, but simply tries to skip at most that amount of bytes. A better name probably would have been trySkip or similar. It seems like most of the time when the argument to skip is small enough, we'll succeed almost all of the time. This is no doubt an implementation artifact of some of the popular stream implementations. This tends to hide the bug-- however, it is still waiting to emerge at some point if those implementations ever change or if buffer sizes are adjusted, etc. All of these cases can be fixed by calling IOUtils#skipFully to get the behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
