[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365335#comment-14365335 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2085 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2085/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365283#comment-14365283 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #135 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/135/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365227#comment-14365227 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2067 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2067/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365188#comment-14365188 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #126 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/126/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364997#comment-14364997 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #135 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/135/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364915#comment-14364915 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-Yarn-trunk #869 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/869/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364903#comment-14364903 ] Rakesh R commented on HDFS-5356: Thanks [~cmccabe] for the reviews. Thanks [~vinayrpet] for reviewing and committing the patch. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364868#comment-14364868 ] Hudson commented on HDFS-5356: -- FAILURE: Integrated in Hadoop-trunk-Commit #7346 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7346/]) HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) (vinayakumarb: rev 018893e81ec1c43e6c79c77adec92c2edfb20cab) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364864#comment-14364864 ] Vinayakumar B commented on HDFS-5356: - Committed to trunk and branch-2. Thanks for the contribution [~rakeshr]. Thanks [~cmccabe] for reviews. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Fix For: 2.8.0 > > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363428#comment-14363428 ] Rakesh R commented on HDFS-5356: Thanks [~vinayrpet] for triggering the build. It looks like the testcase failure is unrelated to this patch. Please have a look at the latest patch, I addressed the review comments. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363045#comment-14363045
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704667/HDFS-5356-8.patch
against trunk revision 3ff1ba2.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9892//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9892//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch,
> HDFS-5356-8.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362451#comment-14362451
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704667/HDFS-5356-8.patch
against trunk revision bc9cb3e.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9886//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch,
> HDFS-5356-8.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362449#comment-14362449 ] Rakesh R commented on HDFS-5356: Re-attaching the same patch to get the jenkins report. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch, > HDFS-5356-8.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361258#comment-14361258
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704235/HDFS-5356-7.patch
against trunk revision 6fdef76.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9880//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359255#comment-14359255
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704235/HDFS-5356-7.patch
against trunk revision b49c3a1.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9860//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356-7.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356652#comment-14356652 ] Vinayakumar B commented on HDFS-5356: - bq. For me all tests in TestRenameWithSnapshots are passing even without the complete patch. Above one was my other account. ;) > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358192#comment-14358192
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704108/HDFS-5356-6.patch
against trunk revision ff83ae7.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9850//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358186#comment-14358186
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704108/HDFS-5356-6.patch
against trunk revision ff83ae7.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9849//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356-6.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356957#comment-14356957
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703881/HDFS-5356-5.patch
against trunk revision 30c428a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9834//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9834//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356724#comment-14356724 ] Rakesh R commented on HDFS-5356: Attached new patch addressing [~vinayrpet]'s comments. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356-5.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356703#comment-14356703 ] Vinayakumar B commented on HDFS-5356: - Yes sure. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356696#comment-14356696
]
Rakesh R commented on HDFS-5356:
OK. Do you agree to go ahead with the changes in {{TestRenameWithSnapshots}} ?
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356690#comment-14356690
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703854/HDFS-5356-4.patch
against trunk revision 30c428a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9831//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9831//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356687#comment-14356687 ] Vinayakumar B commented on HDFS-5356: - bq. It is failing when comparing SnapshotTestHelper.compareDumpedTreeInFile(fsnBefore, fsnMiddle, compareQuota);. fsnBefore and fsnMiddle differs in their storagespace values. This seems unrelated to closure of filesystem. OK now I got. If you are closing the filesystem, then open files are being closed which results in diff in these image sizes. I didnt check that image file sizes were being checked in the test. I just told about the functionality. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356677#comment-14356677
]
Rakesh R commented on HDFS-5356:
It is failing when comparing
{{SnapshotTestHelper.compareDumpedTreeInFile(fsnBefore, fsnMiddle,
compareQuota);}}. fsnBefore and fsnMiddle differs in their {{storagespace}}
values.
{code}
fsnBefore:
Quota[namespace: 4/9223372036854775806, storagespace: 9216/9223372036854775806,
StorageType: DISK-StorageType: SSD-StorageType: ARCHIVE-]
fsnMiddle:
Quota[namespace: 4/9223372036854775806, storagespace: 6147/9223372036854775806,
StorageType: DISK-StorageType: SSD-StorageType: ARCHIVE-]
{code}
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356649#comment-14356649 ] VKB commented on HDFS-5356: --- For me all tests in TestRenameWithSnapshots are passing even without the complete patch. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356-4.patch, HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356522#comment-14356522
]
Rakesh R commented on HDFS-5356:
Thanks a lot [~vinayrpet] for the comments.
bq. Changes in TestRenameWithSnapshots.java are not at all required as hdfs is
re-initialized everytime cluster is restarted. Let it close the FS during
shutdown.
I could see {{TestRenameWithSnapshots#testAppendFileAfterRenameInSnapshot}} is
failing in my environment and with my changes in
{{TestRenameWithSnapshots.java}} this is passing. Could you run and see the
result.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356440#comment-14356440
]
Vinayakumar B commented on HDFS-5356:
-
Hi [~rakeshr], thanks for the patch.
Here are some comments.
Here need to passdown {{deleteDfsDir}}
{code} public void shutdown(boolean deleteDfsDir) {
+shutdown(false, true);
+ }{code}
I Don't think below change are necessary in TestFileCreation.java.
{code}+DistributedFileSystem fs1 = null;{code}
in-fact you can remove the original re-assignment code after cluster restart.
That should ideally work as the NameNode restarted with same ports and data. I
have verified ;)
{code}fs = cluster.getFileSystem();{code}
Changes in TestRenameWithSnapshots.java are not at all required as {{hdfs}} is
re-initialized everytime cluster is restarted. Let it close the FS during
shutdown.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356438#comment-14356438
]
VKB commented on HDFS-5356:
---
Hi [~rakeshr], thanks for the patch.
Here are some comments.
Here need to passdown {{deleteDfsDir}}
{code} public void shutdown(boolean deleteDfsDir) {
+shutdown(false, true);
+ }{code}
I Don't think below change are necessary in TestFileCreation.java.
{code}+DistributedFileSystem fs1 = null;{code}
in-fact you can remove the original re-assignment code after cluster restart.
That should ideally work as the NameNode restarted with same ports and data. I
have verified ;)
{code}fs = cluster.getFileSystem();{code}
Changes in TestRenameWithSnapshots.java are not at all required as {{hdfs}} is
re-initialized everytime cluster is restarted. Let it close the FS during
shutdown.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356413#comment-14356413
]
Rakesh R commented on HDFS-5356:
Thanks [~cmccabe] for the explanation and helps to understand it more.
bq.I would be OK with the change you have now if you checked whether the FS was
in the fileSystems list prior to adding it. Currently it appears like we may
call close twice on the same fs if MiniDFSCluster#getFileSystemInstance was
called more than once.
Yeah, thats good point. I think making the datastructure to {{java.util.Set}}
instead of {{java.util.List}} would be sufficient to avoid the duplicates.
Could you please have a look at the new patch when you get some time.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356-4.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329259#comment-14329259
]
Colin Patrick McCabe commented on HDFS-5356:
Hi Rakesh,
That's a very fair question. I agree that it's nicer in theory to have
{{MiniDFSCluster#shutdown}} call close on every FS.
What happened in practice is that almost all the long-lived resources have
moved out of FileSystem (and its delegate DFSClient) and into ClientContext.
This happened because the FileContext API doesn't have a close function, and a
lot of the code is shared.
FileSystem#close has also created a lot of problems over the years because
while we give out the same FS instance to multiple caller of FileSystem#get, we
don't reference count it (meaning that one thread which calls close on it can
shut it down even if there are other users). This shouldn't be a problem in
unit tests, but it is one reason hadoop FS closing doesn't happen that often in
real systems.
All that being said, I would be OK with the change you have now if you checked
whether the FS was in the fileSystems list prior to adding it. Currently it
appears like we may call close twice on the same fs if
MiniDFSCluster#getFileSystemInstance was called more than once.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325554#comment-14325554
]
Rakesh R commented on HDFS-5356:
Thanks [~cmccabe] for the reply. Yes, that would be fine for these two test
cases. But as the the defect says, should we have a generic way of closing all
the {{filesystems}} during minidfs#shutdown?.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325208#comment-14325208 ] Colin Patrick McCabe commented on HDFS-5356: Can we just change TestFileCreation.java and TestRenameWithSnapshots.java to call FileSystem#close manually? This seems simpler. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323801#comment-14323801
]
Rakesh R commented on HDFS-5356:
[~cmccabe] I've replaced {{FileSystem#closeAll}} with another approach. Kindly
look at the latest patch when you get some time. Thanks!
I think, the test failures are unrelated.
{code}
Tests in error:
TestFSImageWithAcl.setUp:52 java.lang.NoClassDefFoundError:
org/apache/hadoop/io/IOUtils$NullOutputStream
TestFSImageWithAcl.tearDown:58 NullPointerException
{code}
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323782#comment-14323782
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699203/HDFS-5356-3.patch
against trunk revision 9729b24.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9596//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9596//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch,
> HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[ https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323628#comment-14323628 ] Rakesh R commented on HDFS-5356: I could see these two test cases are passing locally. Reattaching the previous patch to get QA report again. > MiniDFSCluster shoud close all open FileSystems when shutdown() > --- > > Key: HDFS-5356 > URL: https://issues.apache.org/jira/browse/HDFS-5356 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: haosdent >Assignee: Rakesh R >Priority: Critical > Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356-3.patch, > HDFS-5356.patch > > > After add some metrics functions to DFSClient, I found that some unit tests > relates to metrics are failed. Because MiniDFSCluster are never close open > FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The > metrics of DFSClients in DefaultMetricsSystem are still exist and this make > other unit tests failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14322909#comment-14322909
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699087/HDFS-5356-2.patch
against trunk revision ab0b958.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9589//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9589//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356-2.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318044#comment-14318044
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698355/HDFS-5356-1.patch
against trunk revision 76e309e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancer
org.apache.hadoop.hdfs.TestFileCreation
org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestGlobPaths
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9558//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9558//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Assignee: Rakesh R
>Priority: Critical
> Attachments: HDFS-5356-1.patch, HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831616#comment-13831616
]
Colin Patrick McCabe commented on HDFS-5356:
I don't think {{FileSystem#closeAll}} is what we want to call here. That will
close all filesystems, regardless of whether they relate to the
{{MiniDFSCluster}}. Remember that {{MiniDFSCluster}} is not a singleton.
Multiple instances can exist.
We might be able to do something smarter here that would close just the
{{FileSystem}} objects that pertain to the {{MiniDFSCluster}}.
However, have you considered just fixing the metrics tests to close the
{{FileSystem}} objects they create? That seems like the simplest thing to do.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.2.0
>Reporter: haosdent
>Priority: Critical
> Attachments: HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800127#comment-13800127
]
Hadoop QA commented on HDFS-5356:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609308/HDFS-5356.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5244//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5244//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: haosdent
>Priority: Critical
> Attachments: HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13797924#comment-13797924
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608917/HDFS-5356.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestFileCreation
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5225//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5225//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: haosdent
>Priority: Critical
> Attachments: HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5356) MiniDFSCluster shoud close all open FileSystems when shutdown()
[
https://issues.apache.org/jira/browse/HDFS-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794443#comment-13794443
]
Hadoop QA commented on HDFS-5356:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12608314/HDFS-5356.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
org.apache.hadoop.hdfs.server.namenode.TestINodeFile
org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
org.apache.hadoop.hdfs.TestDFSClientFailover
org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
org.apache.hadoop.hdfs.web.TestWebHDFSForHA
org.apache.hadoop.hdfs.TestFileCreation
org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5182//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5182//console
This message is automatically generated.
> MiniDFSCluster shoud close all open FileSystems when shutdown()
> ---
>
> Key: HDFS-5356
> URL: https://issues.apache.org/jira/browse/HDFS-5356
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: haosdent
>Priority: Critical
> Attachments: HDFS-5356.patch
>
>
> After add some metrics functions to DFSClient, I found that some unit tests
> relates to metrics are failed. Because MiniDFSCluster are never close open
> FileSystems, DFSClients are alive after MiniDFSCluster shutdown(). The
> metrics of DFSClients in DefaultMetricsSystem are still exist and this make
> other unit tests failed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
