[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255588#comment-15255588 ] Guram Savinov commented on HDFS-2054: - I have this IOException in unit-tests with miniDFS cluster. Spark job writes to HDFS a file which is about 100MB. Could you explain the problem for me: miniDFS dataNode closes socket right after it gets last bytes of the file, but block sender tries to transfer full last block which is greater than last data chunk. Am I right? > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065411#comment-13065411 ] Suresh Srinivas commented on HDFS-2054: --- I missed Jakob's comment as I had not looked though all the comments. I looked at HDFS-2054-2.patch, I still see it. The committed code does not have it though. Just wanted to make sure we are all aware of need to remove StringifyException. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065194#comment-13065194 ] Kihwal Lee commented on HDFS-2054: -- @suresh Is it something different from what Jakob mentioned above? This patch does not use StringifyException as suggested by Jakob and the rest were cleaned up in HDFS-1977. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065102#comment-13065102 ] Suresh Srinivas commented on HDFS-2054: --- Sorry for the late comment. We should cleanup using stringifyException. This could be cleaned up later by a single jira though. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064524#comment-13064524 ] Hudson commented on HDFS-2054: -- Integrated in Hadoop-Hdfs-trunk #723 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/723/]) HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() -- moved the change notice into 0.22 section (i'd originally committed it in trunk section) HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() stack : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1145753 Files : * /hadoop/common/trunk/hdfs/CHANGES.txt stack : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1145751 Files : * /hadoop/common/trunk/hdfs/CHANGES.txt * /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064235#comment-13064235 ] Hudson commented on HDFS-2054: -- Integrated in Hadoop-Hdfs-22-branch #71 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/71/]) HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() stack : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1145752 Files : * /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064129#comment-13064129 ] Hudson commented on HDFS-2054: -- Integrated in Hadoop-Hdfs-trunk-Commit #782 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/782/]) HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() -- moved the change notice into 0.22 section (i'd originally committed it in trunk section) HDFS-2054 BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() stack : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1145753 Files : * /hadoop/common/trunk/hdfs/CHANGES.txt stack : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1145751 Files : * /hadoop/common/trunk/hdfs/CHANGES.txt * /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064116#comment-13064116 ] Jakob Homan commented on HDFS-2054: --- +1 > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064112#comment-13064112 ] Kihwal Lee commented on HDFS-2054: -- bq. -1 tests included. The patch doesn't appear to include any new or modified tests. The previous justification also applies to the new patch. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064104#comment-13064104 ] Hadoop QA commented on HDFS-2054: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486211/HDFS-2054.patch against trunk revision 1145428. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/909//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/909//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/909//console This message is automatically generated. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064064#comment-13064064 ] Kihwal Lee commented on HDFS-2054: -- I incoporated Jakob's comment and a new patch has been posted. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064054#comment-13064054 ] Jakob Homan commented on HDFS-2054: --- Since we're trying to remove the calls to StringifyException (HDFS-1977), can we do so with this patch as well? > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064041#comment-13064041 ] stack commented on HDFS-2054: - +1 on patch. Nice comment. Small change. Will commit in next couple of hours unless objection. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063388#comment-13063388 ] Kihwal Lee commented on HDFS-2054: -- No test was added since the patch only changes the log message. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054613#comment-13054613 ] Hadoop QA commented on HDFS-2054: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483738/HDFS-2054-2.patch against trunk revision 1139124. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/832//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/832//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/832//console This message is automatically generated. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054549#comment-13054549 ] Kihwal Lee commented on HDFS-2054: -- Thanks. I will repost the patch. Last time I did that Jenkins didn't pick it up. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054-2.patch, HDFS-2054.patch, > HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054545#comment-13054545 ] Aaron T. Myers commented on HDFS-2054: -- Hey Kihwal, simply switching from patch available -> open -> patch available won't trigger a new pre-commit test-patch run. You either need to upload a new file (which could have the same contents) or ask someone with Hudson power to kick it for you. Looks like there's presently something missing from my Hudson privs, otherwise I would do it for you. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050072#comment-13050072 ] Kihwal Lee commented on HDFS-2054: -- The new patch has been uploaded. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054-1.patch, HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[
https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050062#comment-13050062
]
Kihwal Lee commented on HDFS-2054:
--
I found the following from Sun's NIO example. :)
j2se/share/sample/nio/server/RequestHandler.java
{code}
} catch (IOException x) {
String m = x.getMessage();
if (!m.equals("Broken pipe") &&
!m.equals("Connection reset by peer")) {
System.err.println("RequestHandler: " + x.toString());
}
{code}
I will add the check.
> BlockSender.sendChunk() prints ERROR for connection closures encountered
> during transferToFully()
> --
>
> Key: HDFS-2054
> URL: https://issues.apache.org/jira/browse/HDFS-2054
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
>Affects Versions: 0.22.0, 0.23.0
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Minor
> Fix For: 0.22.0, 0.23.0
>
> Attachments: HDFS-2054.patch, HDFS-2054.patch
>
>
> The addition of ERROR was part of HDFS-1527. In environments where clients
> tear down FSInputStream/connection before reaching the end of stream, this
> error message often pops up. Since these are not really errors and especially
> not the fault of data node, the message should be toned down at least.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050045#comment-13050045 ] Todd Lipcon commented on HDFS-2054: --- Hey Kihwal. Don't we want to check for "Connection reset by peer" also? I've seen that as well. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049995#comment-13049995 ] Kihwal Lee commented on HDFS-2054: -- Test failures: TestHDFSCLI: The quota related test failures are not due to this patch. They also failed in build #696. TestHDFSTrash : It was failing in other recent pre-commit builds: e.g. https://builds.apache.org/job/PreCommit-HDFS-Build/786/ Tests included: No test is included as justified above. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049977#comment-13049977 ] Hadoop QA commented on HDFS-2054: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12482699/HDFS-2054.patch against trunk revision 1136132. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestHDFSTrash +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/788//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/788//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/788//console This message is automatically generated. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049952#comment-13049952 ] Kihwal Lee commented on HDFS-2054: -- No test is added since it only changes the log message. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Fix For: 0.22.0, 0.23.0 > > Attachments: HDFS-2054.patch, HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047417#comment-13047417 ] Todd Lipcon commented on HDFS-2054: --- Makes sense, let's just do (a) then > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047404#comment-13047404 ] Kihwal Lee commented on HDFS-2054: -- @Todd: (b) may not work either. TransferTo() accesses fd directly and nothing sets the class private variables for indicating the connection status in objects that uses it (e.g. Socket). SocketChannel.write() will also cause EPIPE and an ioe at this point. Only after close() method is explicitly called, the private variables are set and subsequently the open/close check methods will work as expected. Reading may work if the read buffer is not empty even after a write got EPIPE. So testing the connection with read is not reliable either. o.a.h.net.SocketOutputStream.write() used in other portions of sendChunks() does catch IOException and close the stream. That's the source of the isOpen() inconsistency I reported above. The NIO could check for EPIPE and make an AsynchronousCloseException thrown in FileChannel. Otherwise it is very hard to handle different types of exceptions in different ways. For SocketChannel, most Java code assumes IOException means connection closure. The same assumption cannot be made in FileChannel (e.g. HDFS-1527). Short of the support from NIO, (a) seems to be the only option. We could do this in o.a.h.net.SocketOutputStream.transferToFully(), but the hadoop-common class might be used by others and connection closure be handled by catching IOException. So the safest thing we can do at this point is adding (a) to BlockSender. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046910#comment-13046910 ] Todd Lipcon commented on HDFS-2054: --- hrm... that's a pain. I guess our options are (a) parsing exception messages, or (b) passing the Socket object itself to BlockSender such that it can determine whether it's still open. Any other good ideas? > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046907#comment-13046907 ] Kihwal Lee commented on HDFS-2054: -- I tried SocketOutputStream.isOpen() in BlockSender.sendChunk(), but it seems even after EPIPE, isOpen() is not guaranteed to return false. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046808#comment-13046808 ] Kihwal Lee commented on HDFS-2054: -- >I'm not in favor of parsing the exception text for behavior-altering >things. But for deciding whether to log at debug vs warn level, it >seems OK to me. This sounds reasonable. >Another thought is to check something like socket.isInputShutdown() >or socket.isConnected()? Maybe we can assume that any case where we >get an IOE but the socket was then found to be disconnected is OK. >If we had a local IOE with the transferto, the socket would still be up. This is even better, IMO. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046804#comment-13046804 ] Todd Lipcon commented on HDFS-2054: --- Yea, it sucks that Java doesn't give us a way to get at the underlying errno in these cases. For the IOEs thrown by the hadoop-native code in common, we actually have an Errno enum that makes life easy. I'm not in favor of parsing the exception text for behavior-altering things. But for deciding whether to log at debug vs warn level, it seems OK to me. Another thought is to check something like socket.isInputShutdown() or socket.isConnected()? Maybe we can assume that any case where we get an IOE but the socket was then found to be disconnected is OK. If we had a local IOE with the transferto, the socket would still be up. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046801#comment-13046801 ] Kihwal Lee commented on HDFS-2054: -- Last time I tried what you said with EAGAIN in transferTo() in attempt to avoid doing epoll() evey time even before sending anything. Some folks were not thrilled about parsing the text. If it can be done in portable/i14n friendly way and people do not object the idea itself... > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046795#comment-13046795 ] Todd Lipcon commented on HDFS-2054: --- Maybe we can check the exception type and message, and only log warning for unexpected ones? EG "Connection reset by peer" and "Broken pipe" are expected exceptions, but anything else should be logged at WARN level. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046765#comment-13046765 ] Kihwal Lee commented on HDFS-2054: -- To minimum, it will get rid of the annoying stack trace. transferTo() is not exactly making it easy to deal with different exceptions differently. I believe things like EAGAIN was fixed now, but to deal with others you have to parse the error itself, which is rather gross. Ideally we want to deal with EAGAIN, EPIPE, etc. separately and if something else happens print an error message. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046760#comment-13046760 ] stack commented on HDFS-2054: - @Kihwal Do we think this enough to address this issue? I see loads of it running hbase loadings on 0.22. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046754#comment-13046754 ] Kihwal Lee commented on HDFS-2054: -- It may reveal interesting errors in the future, so the log level is being lowered to warn. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Minor > Attachments: HDFS-2054.patch > > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2054) BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()
[ https://issues.apache.org/jira/browse/HDFS-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046725#comment-13046725 ] Patrick Kling commented on HDFS-2054: - If I remember correctly, the bug fixed by HDFS-1527 was causing the affected transfers to fail silently. That's why I added this message. If it is polluting the log file, I have no objection to downgrading this to a warning. > BlockSender.sendChunk() prints ERROR for connection closures encountered > during transferToFully() > -- > > Key: HDFS-2054 > URL: https://issues.apache.org/jira/browse/HDFS-2054 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node >Affects Versions: 0.22.0, 0.23.0 >Reporter: Kihwal Lee >Assignee: Patrick Kling > > The addition of ERROR was part of HDFS-1527. In environments where clients > tear down FSInputStream/connection before reaching the end of stream, this > error message often pops up. Since these are not really errors and especially > not the fault of data node, the message should be toned down at least. > Assigning to the author of HDFS-1527. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
