[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094603#comment-13094603 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Mapreduce-trunk #802 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/802/]) HDFS-1330 and HADOOP-6889. Added additional unit tests. Contributed by John George. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163463 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094222#comment-13094222 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Common-trunk-Commit #811 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/811/]) HDFS-1330 and HADOOP-6889. Added additional unit tests. Contributed by John George. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163463 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094225#comment-13094225 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Hdfs-trunk-Commit #888 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/888/]) HDFS-1330 and HADOOP-6889. Added additional unit tests. Contributed by John George. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163463 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094227#comment-13094227 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Mapreduce-trunk-Commit #821 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/821/]) HDFS-1330 and HADOOP-6889. Added additional unit tests. Contributed by John George. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163463 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094254#comment-13094254 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Hdfs-trunk #777 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/777/]) HDFS-1330 and HADOOP-6889. Added additional unit tests. Contributed by John George. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163463 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092975#comment-13092975 ] John George commented on HDFS-1330: --- TestHost2NodesMap failure is unrelated. It seems to have failed on previous builds as well.. Build #1171 is an example. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092976#comment-13092976 ] Matt Foley commented on HDFS-1330: -- Thank John, it's his patch :-) I just moved it here so test-patch would run on it correctly; it was refusing to load the patch from within HADOOP-6889. Thanks Uma for pointing out the link to HADOOP-7488. It looks like you've got that under control, so we won't need to hold this (HDFS-1330) open for that. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[
https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092526#comment-13092526
]
Uma Maheswara Rao G commented on HDFS-1330:
---
Please check the below code snippet from Datanode, where we are not passing the
rpcTimeOut.
{code}
DatanodeProtocol dnp =
(DatanodeProtocol)RPC.waitForProxy(DatanodeProtocol.class,
DatanodeProtocol.versionID, nnAddr, conf);
{code}
This will leads to the issue..HADOOP-7488.
Make RPCs to DataNodes timeout
--
Key: HDFS-1330
URL: https://issues.apache.org/jira/browse/HDFS-1330
Project: Hadoop HDFS
Issue Type: New Feature
Components: data-node
Affects Versions: 0.22.0, 0.23.0
Reporter: Hairong Kuang
Assignee: John George
Fix For: 0.22.0, 0.23.0, 0.24.0
Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch
This jira aims to make client/datanode or datanode/datanode RPC to have a
timeout of DataNode#socketTimeout.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092409#comment-13092409 ] Hadoop QA commented on HDFS-1330: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12491983/HADOOP-6889-fortrunk-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.blockmanagement.TestHost2NodesMap +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1173//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1173//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1173//console This message is automatically generated. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092410#comment-13092410 ] Hadoop QA commented on HDFS-1330: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12491983/HADOOP-6889-fortrunk-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1174//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1174//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1174//console This message is automatically generated. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092421#comment-13092421 ] Uma Maheswara Rao G commented on HDFS-1330: --- Thanks Matt for the patch. This change is very much required. see the scenario in HADOOP-7488 Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Hairong Kuang Assignee: John George Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HADOOP-6889-fortrunk-2.patch, hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900619#action_12900619 ] Hudson commented on HDFS-1330: -- Integrated in Hadoop-Hdfs-trunk-Commit #370 (See [https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/370/]) Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 Attachments: hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12896684#action_12896684 ] sam rash commented on HDFS-1330: +1 lgtm Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 Attachments: hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12896715#action_12896715 ] Hairong Kuang commented on HDFS-1330: - Thanks Sam for reviewing the patch. Test results ran on my linux box are posted below: ant test-patch: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to i [exec] nclude 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Ant test did not succeed. Failed tests included TestBlockRecovery, TestHDFSTrash(timeout), and TestBackupNode. But they seemed not related to this patch. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 Attachments: hdfsRpcTimeout.patch This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895115#action_12895115 ] Wang Xu commented on HDFS-1330: --- What's the relation between this issue and HDFS-1325 ? Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1330) Make RPCs to DataNodes timeout
[ https://issues.apache.org/jira/browse/HDFS-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12895132#action_12895132 ] Hairong Kuang commented on HDFS-1330: - There are two kinds of communications to a DataNode. Most of them directly uses socket for things like replica reading or writing. A few of them use RPC. HDFS-1325 is a problem with file read/write, while this jira is for a problem with RPC communications. We see that a problematic DataNode causes a client stuck in waiting for response forever. Having a timeout should help with this problem. Make RPCs to DataNodes timeout -- Key: HDFS-1330 URL: https://issues.apache.org/jira/browse/HDFS-1330 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 This jira aims to make client/datanode or datanode/datanode RPC to have a timeout of DataNode#socketTimeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
