[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[
https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986872#comment-14986872
]
Hadoop QA commented on HDFS-9354:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 6s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run
convertXmlToText from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 48s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 38s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 23s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 22s
{color} | {color:red} Patch generated 58 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 26s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_60 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| JDK v1.7.0_79 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.TestParallelShortCircuitReadUnCached |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.7.1 Server=1.7.1
Image:test-patch-base-hadoop-date2015-11-03 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12770211/HDFS-9354.01.patch |
| JIRA Issue | HDFS-9354 |
| Optional Tests | asflicense javac javadoc mvninstall unit findbugs
checkstyle compile |
| uname | Linux a40ec487e282 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987845#comment-14987845 ] Chris Nauroth commented on HDFS-9354: - Something is going wrong with the Findbugs pre-check. I'll need to follow up on that separately. I cannot reproduce any of the test failures, and they are not related to the code changed in this patch. The license check warning is a known issue with HDFS tests writing output files outside of the Maven build root. This is tracked elsewhere. > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988226#comment-14988226 ] Hudson commented on HDFS-9354: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #633 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/633/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988336#comment-14988336 ] Xiaoyu Yao commented on HDFS-9354: -- Thank you, [~cnauroth] for reviewing and committing the patch! > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988248#comment-14988248 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1356 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1356/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988002#comment-14988002 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-trunk-Commit #8748 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8748/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988082#comment-14988082 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2563 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2563/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988456#comment-14988456 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2504 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2504/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988435#comment-14988435 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #622 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/622/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988622#comment-14988622 ] Hudson commented on HDFS-9354: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #567 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/567/]) HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on (cnauroth: rev 095ac834022df6136b42961c507ec745c6cf8f97) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Fix For: 2.8.0 > > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985887#comment-14985887 ] Xiaoyu Yao commented on HDFS-9354: -- Thanks [~cnauroth] for reviewing the patch and providing helpful suggestions. bq. 1. We could add a JUnit @After method that always shuts down cluster if it is non-null. Then, the individual tests wouldn't need to do try-finally, and any new tests that get added over time will get the automatic shutdown for free. This would require a bigger patch though. That's a good idea and I had similar thoughts too. Compared with the small change in patch v0, it would require a bigger patch as you mentioned but can help us avoid leaks in future. I can update the patch based on that. bq. 2. The check for HadoopIllegalArgumentException could be simplified by using JUnit's ExpectedException rule. If you'd like to see a simple example of this, I recommend looking at TestAclConfigFlag. My understanding of "Rule and ExpectedException" (JUnit 4.7) is an alternative to the @Test(expected= HadoopIllegalArgumentException.class), which allows finer grain validation of exception message. But both will need to rely on JUnit @After method to ensure cluster is shutdown upon exception. > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HDFS-9354.00.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[
https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986041#comment-14986041
]
Chris Nauroth commented on HDFS-9354:
-
Thanks, [~xyao].
Yes, you're right about {{ExpectedException}}. On further review of the code,
I don't think this part is really relevant to the current patch. Please
disregard this part of my feedback. I'll review again when the change to use
an {{@After}} cleanup method is available.
> Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
> --
>
> Key: HDFS-9354
> URL: https://issues.apache.org/jira/browse/HDFS-9354
> Project: Hadoop HDFS
> Issue Type: Test
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
> Attachments: HDFS-9354.00.patch
>
>
> This negative test expect HadoopIllegalArgumentException on illegal
> configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class)
> and passed fine on Linux.
> On windows, this test passes as well. But it left open handles on NN metadata
> directories used by MiniDFSCluster. As a result, quite a few of subsequent
> TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents
> them from cleaning up NN metadata directories on Windows.
> This JIRA is opened to explicitly catch the Exception and ensure the test
> cluster is properly shutdown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[
https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986623#comment-14986623
]
Hadoop QA commented on HDFS-9354:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 13s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run
convertXmlToText from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 6s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 19s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 19s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 19s
{color} | {color:red} Patch generated 56 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 169m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests |
hadoop.hdfs.server.datanode.TestBlockScanner |
| | hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock |
| | hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.server.datanode.TestBlockReplacement |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
| JDK v1.7.0_79 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots |
| |
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[ https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986673#comment-14986673 ] Chris Nauroth commented on HDFS-9354: - That was a very strange test-patch run. I submitted another run. Let's see how it goes. > Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows > -- > > Key: HDFS-9354 > URL: https://issues.apache.org/jira/browse/HDFS-9354 > Project: Hadoop HDFS > Issue Type: Test > Components: test >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao > Attachments: HDFS-9354.00.patch, HDFS-9354.01.patch > > > This negative test expect HadoopIllegalArgumentException on illegal > configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class) > and passed fine on Linux. > On windows, this test passes as well. But it left open handles on NN metadata > directories used by MiniDFSCluster. As a result, quite a few of subsequent > TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents > them from cleaning up NN metadata directories on Windows. > This JIRA is opened to explicitly catch the Exception and ensure the test > cluster is properly shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[
https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983975#comment-14983975
]
Hadoop QA commented on HDFS-9354:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 5s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run
convertXmlToText from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 59s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 44s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 52s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 21s
{color} | {color:red} Patch generated 56 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 157m 31s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.hdfs.TestReplaceDatanodeOnFailure |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| JDK v1.7.0_79 Failed junit tests |
hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.7.0 Server=1.7.0
Image:test-patch-base-hadoop-date2015-10-31 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12769933/HDFS-9354.00.patch |
| JIRA Issue | HDFS-9354 |
| Optional Tests | asflicense javac javadoc mvninstall unit findbugs
checkstyle compile |
| uname | Linux a4146648b3cc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
[jira] [Commented] (HDFS-9354) Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
[
https://issues.apache.org/jira/browse/HDFS-9354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14984141#comment-14984141
]
Chris Nauroth commented on HDFS-9354:
-
Hi [~xyao]. Thanks for tracking down these file handle leaks. I have a couple
of thoughts that might help {{TestBalancer}} become more resilient to these
kinds of problems in the future.
# We could add a JUnit {{@After}} method that always shuts down {{cluster}} if
it is non-null. Then, the individual tests wouldn't need to do try-finally,
and any new tests that get added over time will get the automatic shutdown for
free. This would require a bigger patch though.
# The check for {{HadoopIllegalArgumentException}} could be simplified by using
JUnit's {{ExpectedException}} rule. If you'd like to see a simple example of
this, I recommend looking at {{TestAclConfigFlag}}.
Please let me know your thoughts on this. Thanks again!
> Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
> --
>
> Key: HDFS-9354
> URL: https://issues.apache.org/jira/browse/HDFS-9354
> Project: Hadoop HDFS
> Issue Type: Test
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
> Attachments: HDFS-9354.00.patch
>
>
> This negative test expect HadoopIllegalArgumentException on illegal
> configuration. It uses JUnit (expected=HadoopIllegalArgumentException.class)
> and passed fine on Linux.
> On windows, this test passes as well. But it left open handles on NN metadata
> directories used by MiniDFSCluster. As a result, quite a few of subsequent
> TestBalancer unit tests can't start MiniDFSCluster. The open handles prevents
> them from cleaning up NN metadata directories on Windows.
> This JIRA is opened to explicitly catch the Exception and ensure the test
> cluster is properly shutdown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
