[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799005#action_12799005 ] Hudson commented on HDFS-564: - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #94 (See [http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/94/]) > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0, 0.22.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794612#action_12794612 ] Hudson commented on HDFS-564: - Integrated in Hadoop-Hdfs-trunk #182 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/182/]) > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0, 0.22.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794349#action_12794349 ] Hudson commented on HDFS-564: - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #159 (See [http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/159/]) > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0, 0.22.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793502#action_12793502 ] Hudson commented on HDFS-564: - Integrated in Hadoop-Hdfs-trunk-Commit #154 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/154/]) . Adding pipeline tests 17-35. Contributed by Nicholas, Kan, and Hairong. > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0, 0.22.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793494#action_12793494 ] Hairong Kuang commented on HDFS-564: The failed core test is not caused by this patch because this patch adds only fault injection tests. > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793484#action_12793484 ] Hadoop QA commented on HDFS-564: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12428669/pipelineTests2.patch against trunk revision 893039. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/87/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/87/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/87/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/87/console This message is automatically generated. > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch, pipelineTests2.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793379#action_12793379 ] Hadoop QA commented on HDFS-564: +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12428501/pipelineTests1.patch against trunk revision 892941. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/154/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/154/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/154/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/154/console This message is automatically generated. > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793364#action_12793364 ] Tsz Wo (Nicholas), SZE commented on HDFS-564: - Test 17-35 are not found in pipelineTests1.patch. Forgot to add them? > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: 0.21.0 >Reporter: Kan Zhang >Assignee: Hairong Kuang >Priority: Blocker > Fix For: 0.21.0 > > Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch, > pipelineTests1.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[
https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792352#action_12792352
]
Konstantin Boudnik commented on HDFS-564:
-
+1 patch looks good.
I've ran fault injection tests and they are passing except that I see an
intermittent failure in TestFiHFlush. However, it the failures are happening in
the tests which do not expect to find a pipeline when {{hflush()}} is called
after a write within block boundaries. I think these three tests (hFlushFi01_a,
hFlushFi02_a, hFlushFi03_a) need to be revisited.
> Adding pipeline test 17-35
> --
>
> Key: HDFS-564
> URL: https://issues.apache.org/jira/browse/HDFS-564
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.21.0
>Reporter: Kan Zhang
>Assignee: Hairong Kuang
>Priority: Blocker
> Fix For: 0.21.0
>
> Attachments: h564-24.patch, h564-25.patch, pipelineTests.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-564) Adding pipeline test 17-35
[ https://issues.apache.org/jira/browse/HDFS-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762466#action_12762466 ] Kan Zhang commented on HDFS-564: I've attached 2 preliminary patches. h564-24.patch is a patch for the pre-append-merge trunk. This patch changes the behavior of BlockReciever.java in 2 ways. 1) When downstream error happens (ends up in handleMirrorOutError()), the receiver thread no longer interrupts the responder thread, which may lead the responder to behave as if local error has occurred and give the wrong idea to upstream node. 2) The responder will try to read all downstream statuses (up to first ERROR status) before sending its own status and forwarding others to upstream node. If the responder fails to read all downstream statuses it needs, it will mark the next downstream datanode as ERROR. h564-24.patch implements all the tests except 26-28 and 31-33. In the case of test 26-28, I've seen intermittent failures similar to those described in HDFS-101, i.e., when the first datanode sends all statuses to DFSClient and closes the socket, DFSClient isn't able to read those statuses and instead gets a TCP reset. As a result, DFSClient will mistakenly consider the first datanode at fault. In the case of test 31-33, the DFSClient will keep receiving seqno == -1 (keep alive) and hang. h564-25.patch is a quick port of h564-24.patch to the current post-append-merge trunk. Unfortunately many of the tests are failing and I may not have time to investigate it. Hopefully, someone can pick it up from here. > Adding pipeline test 17-35 > -- > > Key: HDFS-564 > URL: https://issues.apache.org/jira/browse/HDFS-564 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Reporter: Kan Zhang >Assignee: Kan Zhang > Attachments: h564-24.patch, h564-25.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
