[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494373#comment-13494373 ] Lars Hofhansl commented on HDFS-3979: - Thanks Nicholas. Luke, so the test you're looking for is starting 3 DNs, and then have the write permanently fail at any of them, and in all cases have the hsync fail on the client, right? Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494379#comment-13494379 ] Luke Lu commented on HDFS-3979: --- bq. So the test you're looking for is starting 3 DNs, and then have the write permanently fail at any of them, and in all cases have the hsync fail on the client, right? Yes, that should provide basic coverage for the failure cases. Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492260#comment-13492260 ] Hudson commented on HDFS-3979: -- Integrated in Hadoop-Yarn-trunk #29 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/29/]) HDFS-3979. For hsync, datanode should wait for the local sync to complete before sending ack. Contributed by Lars Hofhansl (Revision 1406382) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1406382 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492321#comment-13492321 ] Hudson commented on HDFS-3979: -- Integrated in Hadoop-Hdfs-trunk #1219 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1219/]) HDFS-3979. For hsync, datanode should wait for the local sync to complete before sending ack. Contributed by Lars Hofhansl (Revision 1406382) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1406382 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492363#comment-13492363 ] Hudson commented on HDFS-3979: -- Integrated in Hadoop-Mapreduce-trunk #1249 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1249/]) HDFS-3979. For hsync, datanode should wait for the local sync to complete before sending ack. Contributed by Lars Hofhansl (Revision 1406382) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1406382 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3979) Fix hsync semantics
[ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491946#comment-13491946 ] Hudson commented on HDFS-3979: -- Integrated in Hadoop-trunk-Commit #2967 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2967/]) HDFS-3979. For hsync, datanode should wait for the local sync to complete before sending ack. Contributed by Lars Hofhansl (Revision 1406382) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1406382 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Fix hsync semantics --- Key: HDFS-3979 URL: https://issues.apache.org/jira/browse/HDFS-3979 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 2.0.2-alpha Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.3-alpha Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt, hdfs-3979-v4.txt See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has already acknowledged as persisted to a client. Edit: Spelling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
