[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414412#comment-13414412 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1136 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1136/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 2.0.0-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414358#comment-13414358 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1103 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1103/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 2.0.0-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414262#comment-13414262 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2534 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2534/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 2.0.0-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414257#comment-13414257 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2469 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2469/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 2.0.0-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414234#comment-13414234 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2488 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2488/]) HDFS-3492. fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe (Revision 1361449) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1361449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 2.0.0-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414170#comment-13414170 ] Colin Patrick McCabe commented on HDFS-3492: +1 LGTM > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413428#comment-13413428 ] Hadoop QA commented on HDFS-3492: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536311/hdfs-3492.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.common.TestJspHelper +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2809//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2809//console This message is automatically generated. > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch, hdfs-3492.txt > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291772#comment-13291772 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1104 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1104/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291722#comment-13291722 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1071 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1071/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291543#comment-13291543 ] Hadoop QA commented on HDFS-3492: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531333/HDFS-3492.002.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.datanode.TestBPOfferService org.apache.hadoop.test.TestHFSTestCase org.apache.hadoop.test.TestHTestCase +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2617//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2617//console This message is automatically generated. > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch, HDFS-3492.002.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291384#comment-13291384 ] Colin Patrick McCabe commented on HDFS-3492: It looks like it's the change to BlockReadeLocal.java that caused the test failure here. I'm still not sure about why. > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291371#comment-13291371 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2351 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2351/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291314#comment-13291314 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2405 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2405/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291311#comment-13291311 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2332 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2332/]) Revert HDFS-3492 from r1347192: patch broke TestShortCircuitLocalRead (Revision 1347796) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347796 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291005#comment-13291005 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk #1103 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1103/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290937#comment-13290937 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk #1070 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1070/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290615#comment-13290615 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2348 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2348/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290560#comment-13290560 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Hdfs-trunk-Commit #2402 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2402/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290553#comment-13290553 ] Hudson commented on HDFS-3492: -- Integrated in Hadoop-Common-trunk-Commit #2329 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2329/]) HDFS-3492. Fix some misuses of InputStream#skip. Contributed by Colin Patrick McCabe. (Revision 1347192) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347192 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/wsrs/InputStreamEntity.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290527#comment-13290527 ] Todd Lipcon commented on HDFS-3492: --- +1, looks good to me. Thanks Colin. > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3492) fix some misuses of InputStream#skip
[ https://issues.apache.org/jira/browse/HDFS-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288253#comment-13288253 ] Hadoop QA commented on HDFS-3492: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530706/HDFS-3492.001.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2578//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2578//console This message is automatically generated. > fix some misuses of InputStream#skip > > > Key: HDFS-3492 > URL: https://issues.apache.org/jira/browse/HDFS-3492 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Attachments: HDFS-3492.001.patch > > > It seems that we have a few cases where programmers are calling > InputStream#skip and not handling "short skips." Unfortunately, the skip > method is documented and implemented so that it doesn't actually skip the > requested number of bytes, but simply tries to skip at most that amount of > bytes. A better name probably would have been trySkip or similar. > It seems like most of the time when the argument to skip is small enough, > we'll succeed almost all of the time. This is no doubt an implementation > artifact of some of the popular stream implementations. This tends to hide > the bug-- however, it is still waiting to emerge at some point if those > implementations ever change or if buffer sizes are adjusted, etc. > All of these cases can be fixed by calling IOUtils#skipFully to get the > behavior that the programmer expects-- i.e., skipping by the specified amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
